Bridging the Arithmetic Gap: The Cognitive Complexity Benchmark and Financial-PoT for Robust Financial Reasoning
On the map: LLM reasoning → Reasoning for agents, planning and code
Abstract (arXiv)
While Large Language Models excel at semantic tasks, they face a critical bottleneck in financial quantitative reasoning, frequently suffering from "Arithmetic Hallucinations" and a systemic failure mode we term "Cognitive Collapse". To strictly quantify this phenomenon, we introduce the Cognitive Complexity Benchmark (CCB), a robust evaluation framework grounded in a dataset constructed from 95 real-world Chinese A-share annual reports. Unlike traditional datasets, the CCB stratifies financial queries into a three-dimensional taxonomy, Data Source, Mapping Difficulty, and Result Unit, enabling the precise diagnosis of reasoning degradation in high-cognitive-load scenarios. To address these failures, we propose the Iterative Dual-Phase Financial-PoT framework. This neuro-symbolic architecture enforces a strict architectural decoupling: it first isolates semantic variable extraction and logic formulation, then offloads computation to an iterative, self-correcting Python sandbox to ensure deterministic execution. Evaluation on the CCB demonstrates that while standard Chain-of-Thought falters on complex tasks, our approach offers superior robustness, elevating the Qwen3-235B model's average accuracy from 59.7% to 67.3% and achieving gains of up to 10-fold in high-complexity reasoning tasks. These findings suggest that architectural decoupling is a critical enabling factor for improving reliability in financial reasoning tasks, providing a transferable architectural insight for precision-critical domains that require tight alignment between semantic understanding and quantitative computation.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 16 of its 23 references that are in the map. First posted on arXiv 29 Jan 2026.
Builds on
16 of 23 references.
References · 16
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
- 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
- 2022PAL: Program-aided Language Modelscited by 635
- 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
- 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
- 2023Faithful Chain-of-Thought Reasoningcited by 285
- 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
- 2025DeepSeek-OCR: Contexts Optical Compressioncited by 174
4 more
- 2023FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Designcited by 163
- 2024LongGenBench: Benchmarking Long-Form Generation in Long Context LLMscited by 47
- 2025A Survey on Large Language Models with some Insights on their Capabilities and Limitationscited by 41
- 2023Transforming Sentiment Analysis in the Financial Domain with ChatGPTcited by 25
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025FinEval-KR: A Financial Domain Evaluation Framework for Large Language Models' Knowledge and Reasoning
- 2026Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements
- 2025FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging
- 2025FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning
- 2026Evaluating Financial Intelligence in Large Language Models: Benchmarking SuperInvesting AI with LLM Engines
- 2026Neuro-Symbolic Financial Reasoning via Deterministic Fact Ledgers and Adversarial Low-Latency Hallucination Detector
- 2026FinRule-Bench: A Benchmark for Joint Reasoning over Financial Tables and Principles
- 2025XFinBench: Benchmarking LLMs in Complex Financial Problem Solving and Reasoning