mlsanityData to 7 Oct 2026

Bridging the Arithmetic Gap: The Cognitive Complexity Benchmark and Financial-PoT for Robust Financial Reasoning

arXiv 2601.21157 ↗First posted 29 Jan 2026cs.AI cs.CLcited by 1

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

While Large Language Models excel at semantic tasks, they face a critical bottleneck in financial quantitative reasoning, frequently suffering from "Arithmetic Hallucinations" and a systemic failure mode we term "Cognitive Collapse". To strictly quantify this phenomenon, we introduce the Cognitive Complexity Benchmark (CCB), a robust evaluation framework grounded in a dataset constructed from 95 real-world Chinese A-share annual reports. Unlike traditional datasets, the CCB stratifies financial queries into a three-dimensional taxonomy, Data Source, Mapping Difficulty, and Result Unit, enabling the precise diagnosis of reasoning degradation in high-cognitive-load scenarios. To address these failures, we propose the Iterative Dual-Phase Financial-PoT framework. This neuro-symbolic architecture enforces a strict architectural decoupling: it first isolates semantic variable extraction and logic formulation, then offloads computation to an iterative, self-correcting Python sandbox to ensure deterministic execution. Evaluation on the CCB demonstrates that while standard Chain-of-Thought falters on complex tasks, our approach offers superior robustness, elevating the Qwen3-235B model's average accuracy from 59.7% to 67.3% and achieving gains of up to 10-fold in high-complexity reasoning tasks. These findings suggest that architectural decoupling is a critical enabling factor for improving reliability in financial reasoning tasks, providing a transferable architectural insight for precision-critical domains that require tight alignment between semantic understanding and quantitative computation.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 16 of its 23 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

16 of 23 references.

References · 16

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  4. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  5. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  6. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  7. 2022PAL: Program-aided Language Modelscited by 635
  8. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  9. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  10. 2023Faithful Chain-of-Thought Reasoningcited by 285
  11. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
  12. 2025DeepSeek-OCR: Contexts Optical Compressioncited by 174
4 more
  1. 2023FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Designcited by 163
  2. 2024LongGenBench: Benchmarking Long-Form Generation in Long Context LLMscited by 47
  3. 2025A Survey on Large Language Models with some Insights on their Capabilities and Limitationscited by 41
  4. 2023Transforming Sentiment Analysis in the Financial Domain with ChatGPTcited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025FinEval-KR: A Financial Domain Evaluation Framework for Large Language Models' Knowledge and Reasoning
  2. 2026Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements
  3. 2025FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging
  4. 2025FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning
  5. 2026Evaluating Financial Intelligence in Large Language Models: Benchmarking SuperInvesting AI with LLM Engines
  6. 2026Neuro-Symbolic Financial Reasoning via Deterministic Fact Ledgers and Adversarial Low-Latency Hallucination Detector
  7. 2026FinRule-Bench: A Benchmark for Joint Reasoning over Financial Tables and Principles
  8. 2025XFinBench: Benchmarking LLMs in Complex Financial Problem Solving and Reasoning