# FinanceQA: A Benchmark for Evaluating Financial Analysis Capabilities of Large Language Models

arXiv: https://arxiv.org/abs/2501.18062 · HTML: https://mlsanity.com/paper/2501.18062/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jan 2025.

## Built on by (most cited first, 19 of 19)

- [Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents](https://mlsanity.com/paper/2604.06132/) (2026)
- [FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning](https://mlsanity.com/paper/2509.13160/) (2025)
- [SpecForge: A Flexible and Efficient Open-Source Training Framework for Speculative Decoding](https://mlsanity.com/paper/2603.18567/) (2026)
- [On Path to Multimodal Historical Reasoning: HistBench and HistAgent](https://mlsanity.com/paper/2505.20246/) (2025)
- [CryptoBench: A Dynamic Benchmark for Expert-Level Evaluation of LLM Agents in Cryptocurrency](https://mlsanity.com/paper/2512.00417/) (2025)
- [ASDA: Automated Skill Distillation and Adaptation for Financial Reasoning](https://mlsanity.com/paper/2603.16112/) (2026)
- [Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligence](https://mlsanity.com/paper/2608.31075/) (2026)
- [UI-Bench: A Benchmark for Evaluating Design Capabilities of AI Text-to-App Tools](https://mlsanity.com/paper/2508.20410/) (2025)
- [Market-Bench: Evaluating Large Language Models on Introductory Quantitative Trading and Market Dynamics](https://mlsanity.com/paper/2512.12264/) (2025)
- [ReasonAny: Incorporating Reasoning Capability to Any Model via Simple and Effective Model Merging](https://mlsanity.com/paper/2601.05560/) (2026)
- [Evaluating Financial Intelligence in Large Language Models: Benchmarking SuperInvesting AI with LLM Engines](https://mlsanity.com/paper/2603.08704/) (2026)
- [Fin-Bias: Comprehensive Evaluation for LLM Decision-Making under human bias in Finance Domain](https://mlsanity.com/paper/2605.09106/) (2026)
- [LATTICE: Evaluating Decision Support Utility of Crypto Agents](https://mlsanity.com/paper/2604.26235/) (2026)
- [Fine-Grained Benchmark Generation for Comprehensive Evaluation of Foundation Models](https://mlsanity.com/paper/2605.18824/) (2026)
- [The Consistency Dilemma in LLMs: Generator-Evaluator Agreement and Vulnerability to Mistakes](https://mlsanity.com/paper/2606.30653/) (2026)
- [FinInteract: Benchmarking Clarification and Intent Integration in Ambiguous Financial Question Answering](https://mlsanity.com/paper/2609.24002/) (2026)
- [FinFIRST: Benchmarking Search Agents for Financial Information Retrieval, Sourcing and Traceability](https://mlsanity.com/paper/2609.25192/) (2026)
- [PluginRSI: Recursive Improvement of Agent Harnesses with Reusable Plugins](https://mlsanity.com/paper/2609.32423/) (2026)
- [FinAutoRubric: Expert-Guided Automatic Rubric Generation for Evaluating Financial Research Agents](https://mlsanity.com/paper/2609.35744/) (2026)

## Abstract (arXiv)

FinanceQA is a testing suite that evaluates LLMs' performance on complex numerical financial analysis tasks that mirror real-world investment work. Despite recent advances, current LLMs fail to meet the strict accuracy requirements of financial institutions, with models failing approximately 60% of realistic tasks that mimic on-the-job analyses at hedge funds, private equity firms, investment banks, and other financial institutions. The primary challenges include hand-spreading metrics, adhering to standard accounting and corporate valuation conventions, and performing analysis under incomplete information - particularly in multi-step tasks requiring assumption generation. This performance gap highlights the disconnect between existing LLM capabilities and the demands of professional financial analysis that are inadequately tested by current testing architectures. Results show that higher-quality training data is needed to support such tasks, which we experiment with using OpenAI's fine-tuning API. FinanceQA is publicly released at [this https URL](https://huggingface.co/datasets/AfterQuery/FinanceQA).