# FinExam-10K: When Retrieval Helps Financial Reasoning?

arXiv: https://arxiv.org/abs/2608.28155 · HTML: https://mlsanity.com/paper/2608.28155/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 28 Aug 2026.

## Builds on (most cited first, 24 of 24)

- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance](https://mlsanity.com/paper/2306.05443/) (2023)
- [ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering](https://mlsanity.com/paper/2210.03849/) (2022)
- [FinanceMath: Knowledge-Intensive Math Reasoning in Finance Domains](https://mlsanity.com/paper/2311.09797/) (2023)
- [FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging](https://mlsanity.com/paper/2506.05828/) (2025)
- [FinCARDS: Card-Based Analyst Reranking for Financial Document Question Answering](https://mlsanity.com/paper/2601.06992/) (2026)
- [FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning](https://mlsanity.com/paper/2506.02515/) (2025)
- [AgentV-RL: Scaling Reward Modeling with Agentic Verifier](https://mlsanity.com/paper/2604.16004/) (2026)
- [Beyond Outcome Verification: Verifiable Process Reward Models for Structured Reasoning](https://mlsanity.com/paper/2601.17223/) (2026)
- [XFinBench: Benchmarking LLMs in Complex Financial Problem Solving and Reasoning](https://mlsanity.com/paper/2508.15861/) (2025)
- [Finch: Benchmarking Finance & Accounting across Spreadsheet-Centric Enterprise Workflows](https://mlsanity.com/paper/2512.13168/) (2025)
- [RealFin: How Well Do LLMs Reason About Finance When Users Leave Things Unsaid?](https://mlsanity.com/paper/2602.07096/) (2026)
- [FLAME: Financial Large-Language Model Assessment and Metrics Evaluation](https://mlsanity.com/paper/2501.06211/) (2025)
- [Advanced Financial Reasoning at Scale: A Comprehensive Evaluation of Large Language Models on CFA Level III](https://mlsanity.com/paper/2507.02954/) (2025)
- [AlphaQuanter: An End-to-End Tool-Augmented Agentic Reinforcement Learning Framework for Stock Trading](https://mlsanity.com/paper/2510.14264/) (2025)
- [FIRE: A Comprehensive Benchmark for Financial Intelligence and Reasoning Evaluation](https://mlsanity.com/paper/2602.22273/) (2026)
- [Unlocking Data Value in Finance: A Study on Distillation and Difficulty-Aware Training](https://mlsanity.com/paper/2603.07223/) (2026)
- [FINESSE-Bench: A Hierarchical Benchmark Suite for Financial Domain Knowledge and Technical Analysis in Large Language Models](https://mlsanity.com/paper/2605.15482/) (2026)
- [Evaluating Large Language Models for Financial Reasoning: A CFA-Based Benchmark Study](https://mlsanity.com/paper/2509.04468/) (2025)
- [FinToolSyn: A forward synthesis Framework for Financial Tool-Use Dialogue Data with Dynamic Tool Retrieval](https://mlsanity.com/paper/2603.24051/) (2026)
- [Are Large Language Models Reliable Reviewers? A Benchmark for Error Detection in Financial Documents](https://mlsanity.com/paper/2608.12342/) (2026)
- [Reasoning Models Ace the CFA Exams](https://mlsanity.com/paper/2512.08270/) (2025)

## Abstract (arXiv)

Professional financial examinations require models to combine domain knowledge, calculation, and judgment, yet no benchmark covers the full CFA and FRM structure under one protocol. We introduce FinExam-10K, to our knowledge the largest reported English benchmark for this setting, with 10,198 expert-reannotated questions spanning CFA Levels I-III and FRM Parts I-II. We release 5,110 questions and sequester 5,088 for a quarterly maintained leaderboard. To separate coverage from local answerability, we report a 10,198-item Full-Coverage Track and a 7,625-item Context-Complete Reasoning Track, which is the primary basis for claims about reasoning from the supplied record. Across 17 models, the best accuracy is 85.29% overall. On the frozen Hard band, the best score is 34.68% on the Full-Coverage Track and 54.57% on the 372 context-complete items. All 17 models share 47 context-complete failures. Function-RAG and FunctionGraph-RAG rescue hundreds of errors but also overturn many correct answers, producing little or negative net gain. A gate trained only on public data decides from the question and initial response when FunctionGraph-RAG should run. On the 5,088 held-out items, the gate invokes FunctionGraph-RAG for 7.9% of questions and improves accuracy from 70.83% to 71.23% (p = .0446).