# FinRule-Bench: A Benchmark for Joint Reasoning over Financial Tables and Principles arXiv: https://arxiv.org/abs/2603.11339 ยท HTML: https://mlsanity.com/paper/2603.11339/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. - Builds on 23 of its 30 references that are in the map. - First posted on arXiv 11 Mar 2026. ## Placed in - LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/) ## Builds on (most cited first, 23 of 23) - [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024) - [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025) - [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020) - [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021) - [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020) - [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021) - [TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data](https://mlsanity.com/paper/2005.08314/) (2020) - [StructGPT: A General Framework for Large Language Model to Reason over Structured Data](https://mlsanity.com/paper/2305.09645/) (2023) - [Faithful Chain-of-Thought Reasoning](https://mlsanity.com/paper/2301.13379/) (2023) - [Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4](https://mlsanity.com/paper/2304.03439/) (2023) - [ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering](https://mlsanity.com/paper/2210.03849/) (2022) - [Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoning](https://mlsanity.com/paper/2301.13808/) (2023) - [Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey](https://mlsanity.com/paper/2402.17944/) (2024) - [TableLlama: Towards Open Large Generalist Models for Tables](https://mlsanity.com/paper/2311.09206/) (2023) - [Revolutionizing Finance with LLMs: An Overview of Applications and Insights](https://mlsanity.com/paper/2401.11641/) (2024) - [A Survey of Large Language Models in Finance (FinLLMs)](https://mlsanity.com/paper/2402.02315/) (2024) - [Large Language Model for Table Processing: A Survey](https://mlsanity.com/paper/2402.05121/) (2024) - [Neural-Symbolic Reasoning over Knowledge Graphs: A Survey from a Query Perspective](https://mlsanity.com/paper/2412.10390/) (2024) - [MarketSenseAI 2.0: Enhancing Stock Analysis through LLM Agents](https://mlsanity.com/paper/2502.00415/) (2025) - [Better Call CLAUSE: A Discrepancy Benchmark for Auditing LLMs Legal Reasoning Capabilities](https://mlsanity.com/paper/2511.00340/) (2025) - [Utilizing Modern Large Language Models (LLM) for Financial Trend Analysis and Digest Creation](https://mlsanity.com/paper/2510.01225/) (2025) - [Improving Multi-hop Logical Reasoning in Knowledge Graphs with Context-Aware Query Representation Learning](https://mlsanity.com/paper/2406.07034/) (2024) - [Complex Logical Query Answering by Calibrating Knowledge Graph Completion Models](https://mlsanity.com/paper/2410.07165/) (2024) ## Built on by (most cited first, 5 of 5) - [FinBalance: A Multi-Document Accounting Reconciliation Benchmark](https://mlsanity.com/paper/2606.15949/) (2026) - [AUDITFLOW: Executable Symbolic Environments for Structured Financial Reporting Verification](https://mlsanity.com/paper/2606.03031/) (2026) - [FinBoardBench: Benchmarking Dynamic Wealth Management and Strategic Financial Reasoning of LLMs via Board Game Simulations](https://mlsanity.com/paper/2605.27896/) (2026) - [FinRCA-Bench: Benchmarking Evidence Retrieval and Reasoning for Financial AI Systems](https://mlsanity.com/paper/2608.18534/) (2026) - [Vision: Data-Centric Anchoring for Robust and Interpretable Agentic AI](https://mlsanity.com/paper/2609.08216/) (2026) ## Abstract (arXiv) Large language models (LLMs) are increasingly applied to financial analysis, yet their ability to audit structured financial statements under explicit accounting principles remains poorly explored. Existing benchmarks primarily evaluate question answering, numerical reasoning, or anomaly detection on synthetically corrupted data, making it unclear whether models can reliably verify or localize rule compliance on correct financial statements. We introduce FinRule-Bench, a benchmark for evaluating diagnostic completeness in rule-based financial reasoning over real-world financial tables. FinRule-Bench pairs ground-truth financial statements with explicit, human-curated accounting principles and spans four canonical statement types: Balance Sheets, Cash Flow Statements, Income Statements, and Statements of Equity. The benchmark defines three auditing tasks that require progressively stronger reasoning capabilities: (i) rule verification, which tests compliance with a single principle; (ii) rule identification, which requires selecting the violated principle from a provided rule set; and (iii) joint rule diagnosis, which requires detecting and localizing multiple simultaneous violations at the record level. We evaluate LLMs under zero-shot and few-shot prompting, and introduce a causal-counterfactual reasoning protocol that enforces consistency between decisions, explanations, and counterfactual judgments. Across tasks and statement types, we find that while models perform well on isolated rule verification, performance degrades sharply for rule discrimination and multi-violation diagnosis. FinRule-Bench provides a principled and reproducible testbed for studying rule-governed reasoning, diagnostic coverage, and failure modes of LLMs in high-stakes financial analysis.