FinExam-10K: When Retrieval Helps Financial Reasoning?
Not on the map. Why?
Abstract (arXiv)
Professional financial examinations require models to combine domain knowledge, calculation, and judgment, yet no benchmark covers the full CFA and FRM structure under one protocol. We introduce FinExam-10K, to our knowledge the largest reported English benchmark for this setting, with 10,198 expert-reannotated questions spanning CFA Levels I-III and FRM Parts I-II. We release 5,110 questions and sequester 5,088 for a quarterly maintained leaderboard. To separate coverage from local answerability, we report a 10,198-item Full-Coverage Track and a 7,625-item Context-Complete Reasoning Track, which is the primary basis for claims about reasoning from the supplied record. Across 17 models, the best accuracy is 85.29% overall. On the frozen Hard band, the best score is 34.68% on the Full-Coverage Track and 54.57% on the 372 context-complete items. All 17 models share 47 context-complete failures. Function-RAG and FunctionGraph-RAG rescue hundreds of errors but also overturn many correct answers, producing little or negative net gain. A gate trained only on public data decides from the question and initial response when FunctionGraph-RAG should run. On the 5,088 held-out items, the gate invokes FunctionGraph-RAG for 7.9% of questions and improves accuracy from 70.83% to 71.23% (p = .0446).
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 36 references that are in the map. First posted on arXiv 28 Aug 2026.
Builds on
24 of 36 references.
References · 24
- 2023BloombergGPT: A Large Language Model for Financecited by 923
- 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
- 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
- 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 217
- 2022ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answeringcited by 194
- 2023FinanceMath: Knowledge-Intensive Math Reasoning in Finance Domainscited by 39
- 2025FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challengingcited by 27
- 2026FinCARDS: Card-Based Analyst Reranking for Financial Document Question Answeringcited by 16
- 2025FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoningcited by 15
- 2025XFinBench: Benchmarking LLMs in Complex Financial Problem Solving and Reasoningcited by 12
- 2026Beyond Outcome Verification: Verifiable Process Reward Models for Structured Reasoningcited by 12
- 2026AgentV-RL: Scaling Reward Modeling with Agentic Verifiercited by 12
12 more
- 2025Finch: Benchmarking Finance & Accounting across Spreadsheet-Centric Enterprise Workflowscited by 9
- 2026RealFin: How Well Do LLMs Reason About Finance When Users Leave Things Unsaid?cited by 9
- 2025FLAME: Financial Large-Language Model Assessment and Metrics Evaluationcited by 8
- 2025Advanced Financial Reasoning at Scale: A Comprehensive Evaluation of Large Language Models on CFA Level IIIcited by 6
- 2025AlphaQuanter: An End-to-End Tool-Augmented Agentic Reinforcement Learning Framework for Stock Tradingcited by 5
- 2026FIRE: A Comprehensive Benchmark for Financial Intelligence and Reasoning Evaluationcited by 5
- 2026FINESSE-Bench: A Hierarchical Benchmark Suite for Financial Domain Knowledge and Technical Analysis in Large Language Modelscited by 4
- 2025Evaluating Large Language Models for Financial Reasoning: A CFA-Based Benchmark Studycited by 3
- 2026Unlocking Data Value in Finance: A Study on Distillation and Difficulty-Aware Trainingcited by 3
- 2026FinToolSyn: A forward synthesis Framework for Financial Tool-Use Dialogue Data with Dynamic Tool Retrievalcited by 3
- 2026Are Large Language Models Reliable Reviewers? A Benchmark for Error Detection in Financial Documentscited by 2
- 2025Reasoning Models Ace the CFA Examscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Models
- 2025FinEval-KR: A Financial Domain Evaluation Framework for Large Language Models' Knowledge and Reasoning
- 2023Can GPT models be Financial Analysts? An Evaluation of ChatGPT and GPT-4 on mock CFA Exams
- 2024Fine-tuning Smaller Language Models for Question Answering over Financial Documents
- 2026FinFIRST: Benchmarking Search Agents for Financial Information Retrieval, Sourcing and Traceability
- 2023BizBench: A Quantitative Reasoning Benchmark for Business and Finance
- 2026FinTradeBench: A Financial Reasoning Benchmark for LLMs
- 2024FinLLMs: A Framework for Financial Reasoning Dataset Generation with Large Language Models