# FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging

arXiv: https://arxiv.org/abs/2506.05828 · HTML: https://mlsanity.com/paper/2506.05828/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 10 of its 40 references that are in the map.
- First posted on arXiv 6 Jun 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 10 of 10)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [Search-o1: Agentic Search-Enhanced Large Reasoning Models](https://mlsanity.com/paper/2501.05366/) (2025)
- [Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback](https://mlsanity.com/paper/2302.12813/) (2023)
- [Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models](https://mlsanity.com/paper/2501.09686/) (2025)
- [Multi-Step Reasoning with Large Language Models, a Survey](https://mlsanity.com/paper/2407.11511/) (2024)

## Built on by (most cited first, 28 of 28)

- [Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale Thinking Model](https://mlsanity.com/paper/2510.18855/) (2025)
- [PRBench: Large-Scale Expert Rubrics for Evaluating High-Stakes Professional Reasoning](https://mlsanity.com/paper/2511.11562/) (2025)
- [Evaluating LLMs in Finance Requires Explicit Bias Consideration](https://mlsanity.com/paper/2602.14233/) (2026)
- [FinTrust: A Comprehensive Benchmark of Trustworthiness Evaluation in Finance Domain](https://mlsanity.com/paper/2510.15232/) (2025)
- [Fin-RATE: A Real-world Financial Analytics and Tracking Evaluation Benchmark for LLMs on SEC Filings](https://mlsanity.com/paper/2602.07294/) (2026)
- [FinMCP-Bench: Benchmarking LLM Agents for Real-World Financial Tool Use under the Model Context Protocol](https://mlsanity.com/paper/2603.24943/) (2026)
- [FinMMR: Make Financial Numerical Reasoning More Multimodal, Comprehensive, and Challenging](https://mlsanity.com/paper/2508.04625/) (2025)
- [Unlocking Data Value in Finance: A Study on Distillation and Difficulty-Aware Training](https://mlsanity.com/paper/2603.07223/) (2026)
- [FINESSE-Bench: A Hierarchical Benchmark Suite for Financial Domain Knowledge and Technical Analysis in Large Language Models](https://mlsanity.com/paper/2605.15482/) (2026)
- [FrontierFinance: A Challenging Benchmark for Measuring Frontier Intelligence of Finance Agents](https://mlsanity.com/paper/2608.11683/) (2026)
- [FinSafetyBench: Evaluating LLM Safety in Real-World Financial Scenarios](https://mlsanity.com/paper/2605.00706/) (2026)
- [PortBench: A Correlation-Aware, Full-Pipeline Benchmark for LLM-Driven Portfolio Management](https://mlsanity.com/paper/2605.27887/) (2026)
- [Exploring Autonomous Agentic Data Engineering for Model Specialization](https://mlsanity.com/paper/2605.30407/) (2026)
- [Even GPT-5.2 Can't Count to Five: The Case for Zero-Error Horizons in Trustworthy LLMs](https://mlsanity.com/paper/2601.15714/) (2026)
- [Data-Driven Function Calling Improvements in Large Language Model for Online Financial QA](https://mlsanity.com/paper/2604.05387/) (2026)
- [INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Models](https://mlsanity.com/paper/2607.24273/) (2026)
- [QianfanHuijin Technical Report: A Novel Multi-Stage Training Paradigm for Finance Industrial LLMs](https://mlsanity.com/paper/2512.24314/) (2025)
- [BizFinBench.v2: Towards Reliable LLMs in Finance via Real-User Data and Offline/Online Bilingual Evaluation](https://mlsanity.com/paper/2601.06401/) (2026)
- [Less Data Approximates More: Earning Faithful Confidence in High-Stakes Domains](https://mlsanity.com/paper/2604.08454/) (2026)
- [FinVerBench: Benchmark Validity and Calibration in Large Language Model Financial Statement Verification](https://mlsanity.com/paper/2605.29586/) (2026)
- [Fighting Numerical Hallucinations via Data-centric Compilation for Online Financial QA](https://mlsanity.com/paper/2605.31064/) (2026)
- [MoCA-Agent: A Market-of-Claims Code Agent for Financial and Numerical Reasoning](https://mlsanity.com/paper/2606.11537/) (2026)
- [RusFinChain: A Russian Benchmark for Verifiable Chain-of-Thought Reasoning in Finance with Fuzzy-Aligned Evaluation](https://mlsanity.com/paper/2607.01388/) (2026)
- [Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements](https://mlsanity.com/paper/2607.28661/) (2026)
- [FinRiskAtlas: Decision-Aligned Evaluation of Large Language Models for Financial Risk Review](https://mlsanity.com/paper/2608.25325/) (2026)
- [FinExam-10K: When Retrieval Helps Financial Reasoning?](https://mlsanity.com/paper/2608.28155/) (2026)
- [When Tokenization is Secretly Output Supervision](https://mlsanity.com/paper/2609.01386/) (2026)
- [Knowledge or Calculator? Decomposing the Skill Premium in Verifiable Financial Agent Workflows](https://mlsanity.com/paper/2610.03564/) (2026)

## Abstract (arXiv)

We introduce FinanceReasoning, a novel benchmark designed to evaluate the reasoning capabilities of large reasoning models (LRMs) in financial numerical reasoning problems. Compared to existing benchmarks, our work provides three key advancements. (1) Credibility: We update 15.6% of the questions from four public datasets, annotating 908 new questions with detailed Python solutions and rigorously refining evaluation standards. This enables an accurate assessment of the reasoning improvements of LRMs. (2) Comprehensiveness: FinanceReasoning covers 67.8% of financial concepts and formulas, significantly surpassing existing datasets. Additionally, we construct 3,133 Python-formatted functions, which enhances LRMs' financial reasoning capabilities through refined knowledge (e.g., 83.2% rightarrow 91.6% for GPT-4o). (3) Challenge: Models are required to apply multiple financial formulas for precise numerical reasoning on 238 Hard problems. The best-performing model (i.e., OpenAI o1 with PoT) achieves 89.1% accuracy, yet LRMs still face challenges in numerical precision. We demonstrate that combining Reasoner and Programmer models can effectively enhance LRMs' performance (e.g., 83.2% rightarrow 87.8% for DeepSeek-R1). Our work paves the way for future research on evaluating and improving LRMs in domain-specific complex reasoning tasks.