Finance AI tools fail in production not because the underlying models are bad, but because of how they are architected. Real financial data is high-volume, exception-dense, and intolerant of probabilistic outputs. Those characteristics expose the limits of single-prompt approaches quickly.
The demo-to-production gap in finance AI
A tool that looks clean in a sandbox falls apart on a real ledger for predictable reasons. Transaction volumes expand dramatically. Categorisation rules conflict. Subsidiaries use different ledger structures. Exceptions become the norm rather than the edge case.
Systems that performed correctly in a controlled environment start generating inaccurate results, fabricating references, or stalling entirely. The failure mode is not dramatic. It is gradual and hard to catch.
Why real finance data breaks generic large language models
Three characteristics create the problem.
- Volume at the transaction layer. Lean organizations process thousands of monthly transactions across multiple sources. Fitting that into a context window is impractical and produces poor results.
- Exception density. Financial operations hinge on edge cases: reversals, partial payments, intercompany transactions, currency adjustments. Systems optimised for typical scenarios fail precisely where the real value is.
- Deterministic expectations. Financial statements cannot tolerate approximate answers. Finance professionals need reproducible, auditable processes, not interpretive outputs.
What agent-based reasoning fixes
Agent-based reasoning replaces a single comprehensive prompt with a set of connected, specialized agents that each handle a narrow function. One agent retrieves and normalises transactions. Another applies accounting rules. Another validates bank reconciliation. Another synthesises findings.
This architecture delivers three concrete advantages.
- Separation of concerns. Narrow responsibilities with limited inputs contain failures and make errors traceable rather than opaque.
- Determinism where it is needed. Accounting operations run as stable rules, not probabilistic responses. The model provides reasoning; the logic stays consistent.
- Auditability. Every operation generates traceable documentation. Teams review specific agent decisions rather than interpreting black-box outputs.
"Finance does not forgive mistakes."
Why Cortena learned this the hard way
In August 2025 we ran a beta with five early adopters connecting live Xero data. The integration worked. The interfaces held. But the AI engine deteriorated under authentic data volumes in ways we did not see in testing. Rather than patching the surface, we rebuilt the foundation around agent-based reasoning. We relaunched in September 2025 with a narrower initial scope, cash flow analysis, and a much more stable architecture.
What this means for finance teams evaluating AI tools
Three questions cut through the noise when you are evaluating finance AI.
- Does it work on your real data? Request a test against your actual transaction volume, not a simplified example.
- What happens when it is wrong? A tool that fails silently is worse than one that flags exceptions clearly.
- Is it one model or many agents? The architecture matters more than the model name on the packaging.
Each agent: narrow inputs · stable rules · traceable output