# Towards reducing hallucination in extracting information from financial reports using Large Language Models

arXiv: https://arxiv.org/abs/2310.10760 · HTML: https://mlsanity.com/paper/2310.10760/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [ToolQA: A Dataset for LLM Question Answering with External Tools](https://mlsanity.com/paper/2306.13304/) (2023)
- [Graph Convolution for Multimodal Information Extraction from Visually Rich Documents](https://mlsanity.com/paper/1903.11279/) (2019)
- [DocILE Benchmark for Document Information Localization and Extraction](https://mlsanity.com/paper/2302.05658/) (2023)
- [GraphIE: A Graph-Based Framework for Information Extraction](https://mlsanity.com/paper/1810.13083/) (2018)
- [RETA-LLM: A Retrieval-Augmented Large Language Model Toolkit](https://mlsanity.com/paper/2306.05212/) (2023)
- [Information Extraction from Scanned Invoice Images using Text Analysis and Layout Features](https://mlsanity.com/paper/2208.04011/) (2022)
- [Paradigm Shift in Sustainability Disclosure Analysis: Empowering Stakeholders with CHATREPORT, a Language Model-Based Tool](https://mlsanity.com/paper/2306.15518/) (2023)
- [Abstractive Information Extraction from Scanned Invoices (AIESI) using End-to-end Sequential Approach](https://mlsanity.com/paper/2009.05728/) (2020)
- [Intelligent Document Processing -- Methods and Tools in the real world](https://mlsanity.com/paper/2112.14070/) (2021)

## Built on by (most cited first, 11 of 11)

- [HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction](https://mlsanity.com/paper/2408.04948/) (2024)
- [UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysis](https://mlsanity.com/paper/2406.15187/) (2024)
- [FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.15800/) (2025)
- [FAITH: A Framework for Assessing Intrinsic Tabular Hallucinations in Finance](https://mlsanity.com/paper/2508.05201/) (2025)
- [Establishing Performance Baselines in Fine-Tuning, Retrieval-Augmented Generation and Soft-Prompting for Non-Specialist LLM Users](https://mlsanity.com/paper/2311.05903/) (2023)
- [Bridging Language Models and Financial Analysis](https://mlsanity.com/paper/2503.22693/) (2025)
- [ChatSchema: A pipeline of extracting structured information with Large Multimodal Models based on schema](https://mlsanity.com/paper/2407.18716/) (2024)
- [Efficacy of Large Language Models in Systematic Reviews](https://mlsanity.com/paper/2408.04646/) (2024)
- [Bypassing Document Ingestion: An MCP Approach to Financial Q&A](https://mlsanity.com/paper/2603.20316/) (2026)
- [A Concise Review of Hallucinations in LLMs and their Mitigation](https://mlsanity.com/paper/2512.02527/) (2025)
- [MimirRAG: A Multi-Agent RAG Framework for Financial Data Retrieval with Metadata Integration](https://mlsanity.com/paper/2605.25030/) (2026)

## Abstract (arXiv)

For a financial analyst, the question and answer (Q&A) segment of the company financial report is a crucial piece of information for various analysis and investment decisions. However, extracting valuable insights from the Q&A section has posed considerable challenges as the conventional methods such as detailed reading and note-taking lack scalability and are susceptible to human errors, and Optical Character Recognition (OCR) and similar techniques encounter difficulties in accurately processing unstructured transcript text, often missing subtle linguistic nuances that drive investor decisions. Here, we demonstrate the utilization of Large Language Models (LLMs) to efficiently and rapidly extract information from earnings report transcripts while ensuring high accuracy transforming the extraction process as well as reducing hallucination by combining retrieval-augmented generation technique as well as metadata. We evaluate the outcomes of various LLMs with and without using our proposed approach based on various objective metrics for evaluating Q&A systems, and empirically demonstrate superiority of our method.