# BizBench: A Quantitative Reasoning Benchmark for Business and Finance

arXiv: https://arxiv.org/abs/2311.06602 · HTML: https://mlsanity.com/paper/2311.06602/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 14 of its 47 references that are in the map.
- First posted on arXiv 11 Nov 2023.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 14 of 14)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://mlsanity.com/paper/2306.08568/) (2023)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation](https://mlsanity.com/paper/2211.11501/) (2022)
- [PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance](https://mlsanity.com/paper/2306.05443/) (2023)
- [BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmark](https://mlsanity.com/paper/2302.09432/) (2023)

## Built on by (most cited first, 28 of 28)

- [FinBen: A Holistic Financial Benchmark for Large Language Models](https://mlsanity.com/paper/2402.12659/) (2024)
- [A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges](https://mlsanity.com/paper/2406.11903/) (2024)
- [No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding](https://mlsanity.com/paper/2503.05061/) (2025)
- [XFinBench: Benchmarking LLMs in Complex Financial Problem Solving and Reasoning](https://mlsanity.com/paper/2508.15861/) (2025)
- [Python is Not Always the Best Choice: Embracing Multilingual Program of Thoughts](https://mlsanity.com/paper/2402.10691/) (2024)
- [FLAME: Financial Large-Language Model Assessment and Metrics Evaluation](https://mlsanity.com/paper/2501.06211/) (2025)
- [TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law](https://mlsanity.com/paper/2507.21134/) (2025)
- [FAITH: A Framework for Assessing Intrinsic Tabular Hallucinations in Finance](https://mlsanity.com/paper/2508.05201/) (2025)
- [FinMMR: Make Financial Numerical Reasoning More Multimodal, Comprehensive, and Challenging](https://mlsanity.com/paper/2508.04625/) (2025)
- [FinLLMs: A Framework for Financial Reasoning Dataset Generation with Large Language Models](https://mlsanity.com/paper/2401.10744/) (2024)
- [Bridging Language Models and Financial Analysis](https://mlsanity.com/paper/2503.22693/) (2025)
- [MBABench: Evaluating LLM Agents on End-to-End Spreadsheet Tasks in Finance](https://mlsanity.com/paper/2605.22664/) (2026)
- [Extract, Match, and Score: An Evaluation Paradigm for Long Question-context-answer Triplets in Financial Analysis](https://mlsanity.com/paper/2503.16575/) (2025)
- [CARGO: A Framework for Confidence-Aware Routing of Large Language Models](https://mlsanity.com/paper/2509.14899/) (2025)
- [FrontierFinance: A Long-Horizon Computer-Use Benchmark of Real-World Financial Tasks](https://mlsanity.com/paper/2604.05912/) (2026)
- [The Price of Agreement: Measuring LLM Sycophancy in Agentic Financial Applications](https://mlsanity.com/paper/2604.24668/) (2026)
- [The CLEF-2026 FinMMEval Lab: Multilingual and Multimodal Evaluation of Financial AI Systems](https://mlsanity.com/paper/2602.10886/) (2026)
- [Cost-Efficient Estimation of General Abilities Across Benchmarks](https://mlsanity.com/paper/2604.01418/) (2026)
- [TeleEval-OS: Performance evaluations of large language models for operations scheduling](https://mlsanity.com/paper/2506.11017/) (2025)
- [AuditAgent: Expert-Guided Multi-Agent Reasoning for Cross-Document Fraudulent Evidence Discovery](https://mlsanity.com/paper/2510.00156/) (2025)
- [Towards Competent AI for Fundamental Analysis in Finance: A Benchmark Dataset and Evaluation](https://mlsanity.com/paper/2506.07315/) (2025)
- [RusFinChain: A Russian Benchmark for Verifiable Chain-of-Thought Reasoning in Finance with Fuzzy-Aligned Evaluation](https://mlsanity.com/paper/2607.01388/) (2026)
- [Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning](https://mlsanity.com/paper/2607.16057/) (2026)
- [Language-Routed RAG and Direct Option Scoring for Multilingual Financial QA: DS@GT at FinMMEval](https://mlsanity.com/paper/2607.22841/) (2026)
- [Evaluating Investment Logic in Large Language Models: A Real-World Benchmark Towards Personalzied Financial Agents](https://mlsanity.com/paper/2608.06108/) (2026)
- [V-FiLLM: Verified Financial LLM Reasoning Benchmark](https://mlsanity.com/paper/2608.11047/) (2026)
- [FinRiskAtlas: Decision-Aligned Evaluation of Large Language Models for Financial Risk Review](https://mlsanity.com/paper/2608.25325/) (2026)
- [RetailAgent: Structured Adverse Timing in Self-Conditioned Multimodal LLM Trading Agents](https://mlsanity.com/paper/2608.28399/) (2026)

## Abstract (arXiv)

Answering questions within business and finance requires reasoning, precision, and a wide-breadth of technical knowledge. Together, these requirements make this domain difficult for large language models (LLMs). We introduce BizBench, a benchmark for evaluating models' ability to reason about realistic financial problems. BizBench comprises eight quantitative reasoning tasks, focusing on question-answering (QA) over financial data via program synthesis. We include three financially-themed code-generation tasks from newly collected and augmented QA data. Additionally, we isolate the reasoning capabilities required for financial QA: reading comprehension of financial text and tables for extracting intermediate values, and understanding financial concepts and formulas needed to calculate complex solutions. Collectively, these tasks evaluate a model's financial background knowledge, ability to parse financial documents, and capacity to solve problems with code. We conduct an in-depth evaluation of open-source and commercial LLMs, comparing and contrasting the behavior of code-focused and language-focused models. We demonstrate that the current bottleneck in performance is due to LLMs' limited business and financial understanding, highlighting the value of a challenging benchmark for quantitative reasoning within this domain.