# FAITH: A Framework for Assessing Intrinsic Tabular Hallucinations in Finance

arXiv: https://arxiv.org/abs/2508.05201 · HTML: https://mlsanity.com/paper/2508.05201/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 41 references that are in the map.
- First posted on arXiv 7 Aug 2025.

## Builds on (most cited first, 25 of 25)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2305.11747/) (2023)
- [RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models](https://mlsanity.com/paper/2401.00396/) (2023)
- [TableBench: A Comprehensive and Complex Benchmark for Table Question Answering](https://mlsanity.com/paper/2408.09174/) (2024)
- [MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Data](https://mlsanity.com/paper/2206.01347/) (2022)
- [HalluLens: LLM Hallucination Benchmark](https://mlsanity.com/paper/2504.17550/) (2025)
- [FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"](https://mlsanity.com/paper/2410.03727/) (2024)
- [Deficiency of Large Language Models in Finance: An Empirical Examination of Hallucination](https://mlsanity.com/paper/2311.15548/) (2023)
- [ReEval: Automatic Hallucination Evaluation for Retrieval-Augmented Large Language Models via Transferable Adversarial Attacks](https://mlsanity.com/paper/2310.12516/) (2023)
- [BizBench: A Quantitative Reasoning Benchmark for Business and Finance](https://mlsanity.com/paper/2311.06602/) (2023)
- [HalluDial: A Large-Scale Benchmark for Automatic Dialogue-Level Hallucination Evaluation](https://mlsanity.com/paper/2406.07070/) (2024)
- [ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Models](https://mlsanity.com/paper/2407.04693/) (2024)
- [RealHiTBench: A Comprehensive Realistic Hierarchical Table Benchmark for Evaluating LLM-Based Table Analysis](https://mlsanity.com/paper/2506.13405/) (2025)
- [Towards reducing hallucination in extracting information from financial reports using Large Language Models](https://mlsanity.com/paper/2310.10760/) (2023)
- [Hallucination-minimized Data-to-answer Framework for Financial Decision-makers](https://mlsanity.com/paper/2311.07592/) (2023)
- [ERBench: An Entity-Relationship based Automatically Verifiable Hallucination Benchmark for Large Language Models](https://mlsanity.com/paper/2403.05266/) (2024)
- [ECC Analyzer: Extract Trading Signal from Earnings Conference Calls using Large Language Model for Stock Performance Prediction](https://mlsanity.com/paper/2404.18470/) (2024)
- [Beyond the Reported Cutoff: Where Large Language Models Fall Short on Financial Knowledge](https://mlsanity.com/paper/2504.00042/) (2025)
- [Journey of Hallucination-minimized Generative AI Solutions for Financial Decision Makers](https://mlsanity.com/paper/2311.10961/) (2023)

## Built on by (most cited first, 8 of 8)

- [Fin-RATE: A Real-world Financial Analytics and Tracking Evaluation Benchmark for LLMs on SEC Filings](https://mlsanity.com/paper/2602.07294/) (2026)
- [Hallucination Detection and Mitigation in Large Language Models](https://mlsanity.com/paper/2601.09929/) (2026)
- [FinGround: Detecting and Grounding Financial Hallucinations via Atomic Claim Verification](https://mlsanity.com/paper/2604.23588/) (2026)
- [Evaluating Financial Intelligence in Large Language Models: Benchmarking SuperInvesting AI with LLM Engines](https://mlsanity.com/paper/2603.08704/) (2026)
- [QO-Bench: Diagnosing Query-Operator-Preserving Retrieval over Typed Event Tuples](https://mlsanity.com/paper/2606.04646/) (2026)
- [Personalized Chain-of-Thought Summarization of Financial News for Investor Decision Support](https://mlsanity.com/paper/2511.05508/) (2025)
- [FinVerBench: Benchmark Validity and Calibration in Large Language Model Financial Statement Verification](https://mlsanity.com/paper/2605.29586/) (2026)
- [AgentFinVQA: A Deployable Multi-Agent Pipeline for Auditable Financial Chart QA](https://mlsanity.com/paper/2606.19782/) (2026)

## Abstract (arXiv)

Hallucination remains a critical challenge for deploying Large Language Models (LLMs) in finance. Accurate extraction and precise calculation from tabular data are essential for reliable financial analysis, since even minor numerical errors can undermine decision-making and regulatory compliance. Financial applications have unique requirements, often relying on context-dependent, numerical, and proprietary tabular data that existing hallucination benchmarks rarely capture. In this study, we develop a rigorous and scalable framework for evaluating intrinsic hallucinations in financial LLMs, conceptualized as a context-aware masked span prediction task over real-world financial documents. Our main contributions are: (1) a novel, automated dataset creation paradigm using a masking strategy; (2) a new hallucination evaluation dataset derived from S&P 500 annual reports; and (3) a comprehensive evaluation of intrinsic hallucination patterns in state-of-the-art LLMs on financial tabular data. Our work provides a robust methodology for in-house LLM evaluation and serves as a critical step toward building more trustworthy and reliable financial Generative AI systems.