FinRule-Bench: A Benchmark for Joint Reasoning over Financial Tables and Principles
On the map: LLM reasoning → Reasoning for agents, planning and code
Abstract (arXiv)
Large language models (LLMs) are increasingly applied to financial analysis, yet their ability to audit structured financial statements under explicit accounting principles remains poorly explored. Existing benchmarks primarily evaluate question answering, numerical reasoning, or anomaly detection on synthetically corrupted data, making it unclear whether models can reliably verify or localize rule compliance on correct financial statements. We introduce FinRule-Bench, a benchmark for evaluating diagnostic completeness in rule-based financial reasoning over real-world financial tables. FinRule-Bench pairs ground-truth financial statements with explicit, human-curated accounting principles and spans four canonical statement types: Balance Sheets, Cash Flow Statements, Income Statements, and Statements of Equity. The benchmark defines three auditing tasks that require progressively stronger reasoning capabilities: (i) rule verification, which tests compliance with a single principle; (ii) rule identification, which requires selecting the violated principle from a provided rule set; and (iii) joint rule diagnosis, which requires detecting and localizing multiple simultaneous violations at the record level. We evaluate LLMs under zero-shot and few-shot prompting, and introduce a causal-counterfactual reasoning protocol that enforces consistency between decisions, explanations, and counterfactual judgments. Across tasks and statement types, we find that while models perform well on isolated rule verification, performance degrades sharply for rule discrimination and multi-violation diagnosis. FinRule-Bench provides a principled and reproducible testbed for studying rule-governed reasoning, diagnostic coverage, and failure modes of LLMs in high-stakes financial analysis.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 23 of its 30 references that are in the map. First posted on arXiv 11 Mar 2026.
Builds on
23 of 30 references.
Closest ancestors · 2
Same direction, newest first
- 2025Better Call CLAUSE: A Discrepancy Benchmark for Auditing LLMs Legal Reasoning Capabilities1 Nov
- 2022ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering7 Oct
Other references · 21
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2020Knowledge Graphscited by 619
- 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
- 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
- 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
- 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
- 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339
- 2023Faithful Chain-of-Thought Reasoningcited by 285
- 2023Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4cited by 202
- 2023Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoningcited by 177
- 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
9 more
- 2023TableLlama: Towards Open Large Generalist Models for Tablescited by 136
- 2024Revolutionizing Finance with LLMs: An Overview of Applications and Insightscited by 88
- 2024A Survey of Large Language Models in Finance (FinLLMs)cited by 77
- 2024Large Language Model for Table Processing: A Surveycited by 52
- 2024Neural-Symbolic Reasoning over Knowledge Graphs: A Survey from a Query Perspectivecited by 20
- 2025MarketSenseAI 2.0: Enhancing Stock Analysis through LLM Agentscited by 16
- 2025Utilizing Modern Large Language Models (LLM) for Financial Trend Analysis and Digest Creationcited by 4
- 2024Improving Multi-hop Logical Reasoning in Knowledge Graphs with Context-Aware Query Representation Learningcited by 3
- 2024Complex Logical Query Answering by Calibrating Knowledge Graph Completion Modelscited by 1
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026FinBalance: A Multi-Document Accounting Reconciliation Benchmarkcited by 3
- 2026AUDITFLOW: Executable Symbolic Environments for Structured Financial Reporting Verificationcited by 1
- 2026FinBoardBench: Benchmarking Dynamic Wealth Management and Strategic Financial Reasoning of LLMs via Board Game Simulationscited by 0
- 2026FinRCA-Bench: Benchmarking Evidence Retrieval and Reasoning for Financial AI Systemscited by 0
- 2026Vision: Data-Centric Anchoring for Robust and Interpretable Agentic AIcited by 0
- 2026Vision: Data-Centric Anchoring for Robust and Interpretable Agentic AI8 Sep 2026
- 2026FinRCA-Bench: Benchmarking Evidence Retrieval and Reasoning for Financial AI Systems19 Aug 2026
- 2026FinBalance: A Multi-Document Accounting Reconciliation Benchmark14 Jun 2026
- 2026AUDITFLOW: Executable Symbolic Environments for Structured Financial Reporting Verification2 Jun 2026
- 2026FinBoardBench: Benchmarking Dynamic Wealth Management and Strategic Financial Reasoning of LLMs via Board Game Simulations27 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements
- 2023BizBench: A Quantitative Reasoning Benchmark for Business and Finance
- 2025FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging
- 2026Are Large Language Models Reliable Reviewers? A Benchmark for Error Detection in Financial Documents
- 2023Beyond Classification: Financial Reasoning in State-of-the-Art Language Models
- 2025FinAuditing: A Financial Taxonomy-Structured Multi-Document Benchmark for Evaluating LLMs
- 2025FinEval-KR: A Financial Domain Evaluation Framework for Large Language Models' Knowledge and Reasoning
- 2026V-FiLLM: Verified Financial LLM Reasoning Benchmark