mlsanityData to 14 Oct 2026

BizBench: A Quantitative Reasoning Benchmark for Business and Finance

arXiv 2311.06602 ↗First posted 11 Nov 2023cs.CLcited by 28

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Answering questions within business and finance requires reasoning, precision, and a wide-breadth of technical knowledge. Together, these requirements make this domain difficult for large language models (LLMs). We introduce BizBench, a benchmark for evaluating models' ability to reason about realistic financial problems. BizBench comprises eight quantitative reasoning tasks, focusing on question-answering (QA) over financial data via program synthesis. We include three financially-themed code-generation tasks from newly collected and augmented QA data. Additionally, we isolate the reasoning capabilities required for financial QA: reading comprehension of financial text and tables for extracting intermediate values, and understanding financial concepts and formulas needed to calculate complex solutions. Collectively, these tasks evaluate a model's financial background knowledge, ability to parse financial documents, and capacity to solve problems with code. We conduct an in-depth evaluation of open-source and commercial LLMs, comparing and contrasting the behavior of code-focused and language-focused models. We demonstrate that the current bottleneck in performance is due to LLMs' limited business and financial understanding, highlighting the value of a challenging benchmark for quantitative reasoning within this domain.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 14 of its 47 references that are in the map. First posted on arXiv 11 Nov 2023.

Builds on

14 of 47 references.

References · 14

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2021Program Synthesis with Large Language Modelscited by 3,637
  6. 2023Mistral 7Bcited by 3,164
  7. 2023BloombergGPT: A Large Language Model for Financecited by 939
  8. 2023StarCoder: may the source be with you!cited by 799
  9. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  10. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  11. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  12. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 452
2 more
  1. 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 222
  2. 2023BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmarkcited by 60

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 12; 2026: 12.

28 papers.

Sort:

By area:

  1. 2024FinBen: A Holistic Financial Benchmark for Large Language Modelscited by 141
  2. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 90
  3. 2025No Free Labels: Limitations of LLM-as-a-Judge Without Human Groundingcited by 54
  4. 2025XFinBench: Benchmarking LLMs in Complex Financial Problem Solving and Reasoningcited by 12
  5. 2024Python is Not Always the Best Choice: Embracing Multilingual Program of Thoughtscited by 8
  6. 2025FLAME: Financial Large-Language Model Assessment and Metrics Evaluationcited by 8
  7. 2025TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Lawcited by 8
  8. 2025FAITH: A Framework for Assessing Intrinsic Tabular Hallucinations in Financecited by 8
  9. 2025FinMMR: Make Financial Numerical Reasoning More Multimodal, Comprehensive, and Challengingcited by 7
  10. 2024FinLLMs: A Framework for Financial Reasoning Dataset Generation with Large Language Modelscited by 6
  11. 2025Bridging Language Models and Financial Analysiscited by 4
  12. 2026MBABench: Evaluating LLM Agents on End-to-End Spreadsheet Tasks in Financecited by 4
16 more
  1. 2025Extract, Match, and Score: An Evaluation Paradigm for Long Question-context-answer Triplets in Financial Analysiscited by 3
  2. 2025CARGO: A Framework for Confidence-Aware Routing of Large Language Modelscited by 3
  3. 2026FrontierFinance: A Long-Horizon Computer-Use Benchmark of Real-World Financial Taskscited by 3
  4. 2026The Price of Agreement: Measuring LLM Sycophancy in Agentic Financial Applicationscited by 3
  5. 2026The CLEF-2026 FinMMEval Lab: Multilingual and Multimodal Evaluation of Financial AI Systemscited by 2
  6. 2026Cost-Efficient Estimation of General Abilities Across Benchmarkscited by 2
  7. 2025TeleEval-OS: Performance evaluations of large language models for operations schedulingcited by 1
  8. 2025AuditAgent: Expert-Guided Multi-Agent Reasoning for Cross-Document Fraudulent Evidence Discoverycited by 1
  9. 2025Towards Competent AI for Fundamental Analysis in Finance: A Benchmark Dataset and Evaluationcited by 0
  10. 2026RusFinChain: A Russian Benchmark for Verifiable Chain-of-Thought Reasoning in Finance with Fuzzy-Aligned Evaluationcited by 0
  11. 2026Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoningcited by 0
  12. 2026Language-Routed RAG and Direct Option Scoring for Multilingual Financial QA: DS@GT at FinMMEvalcited by 0
  13. 2026Evaluating Investment Logic in Large Language Models: A Real-World Benchmark Towards Personalzied Financial Agentscited by 0
  14. 2026V-FiLLM: Verified Financial LLM Reasoning Benchmarkcited by 0
  15. 2026FinRiskAtlas: Decision-Aligned Evaluation of Large Language Models for Financial Risk Reviewcited by 0
  16. 2026RetailAgent: Structured Adverse Timing in Self-Conditioned Multimodal LLM Trading Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025BizFinBench: A Business-Driven Real-World Financial Benchmark for Evaluating LLMs
  2. 2025Evaluating Large Language Models for Financial Reasoning: A CFA-Based Benchmark Study
  3. 2025FinEval-KR: A Financial Domain Evaluation Framework for Large Language Models' Knowledge and Reasoning
  4. 2025FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging
  5. 2026From Documents to Reasoning: A Validated Synthetic Data Pipeline and Semantic-Aware Fine-Tuning for Financial Numerical Reasoning
  6. 2026FinTradeBench: A Financial Reasoning Benchmark for LLMs
  7. 2026QuantEval: A Benchmark for Financial Quantitative Tasks in Large Language Models
  8. 2023Beyond Classification: Financial Reasoning in State-of-the-Art Language Models