# Improving Retrieval for RAG based Question Answering Models on Financial Documents

arXiv: https://arxiv.org/abs/2404.07221 · HTML: https://mlsanity.com/paper/2404.07221/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 4 of its 7 references that are in the map.
- First posted on arXiv 23 Mar 2024.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 4 of 4)

- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs](https://mlsanity.com/paper/2312.05934/) (2023)
- [Financial Report Chunking for Effective Retrieval Augmented Generation](https://mlsanity.com/paper/2402.05131/) (2024)

## Built on by (most cited first, 30 of 49)

- [A Survey on Knowledge-Oriented Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.10677/) (2025)
- [KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Model](https://mlsanity.com/paper/2506.20923/) (2025)
- [Toolshed: Scale Tool-Equipped Agents with Advanced RAG-Tool Fusion and Tool Knowledge Bases](https://mlsanity.com/paper/2410.14594/) (2024)
- [FinSage: A Multi-aspect RAG System for Financial Filings Question Answering](https://mlsanity.com/paper/2504.14493/) (2025)
- [Silent Leaks: Implicit Knowledge Extraction Attack on RAG Systems through Benign Queries](https://mlsanity.com/paper/2505.15420/) (2025)
- [FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.15800/) (2025)
- [MarketSenseAI 2.0: Enhancing Stock Analysis through LLM Agents](https://mlsanity.com/paper/2502.00415/) (2025)
- [FACTS About Building Retrieval Augmented Generation-based Chatbots](https://mlsanity.com/paper/2407.07858/) (2024)
- [Graph RAG-Tool Fusion](https://mlsanity.com/paper/2502.07223/) (2025)
- [Multi-Reranker: Maximizing performance of retrieval-augmented generation in the FinanceRAG challenge](https://mlsanity.com/paper/2411.16732/) (2024)
- [Understanding the Impact of Confidence in Retrieval Augmented Generation: A Case Study in the Medical Domain](https://mlsanity.com/paper/2412.20309/) (2024)
- [ASRank: Zero-Shot Re-Ranking with Answer Scent for Document Retrieval](https://mlsanity.com/paper/2501.15245/) (2025)
- [Systematic Knowledge Injection into Large Language Models via Diverse Augmentation for Domain-Specific RAG](https://mlsanity.com/paper/2502.08356/) (2025)
- [Optimizing Retrieval Strategies for Financial Question Answering Documents in Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2503.15191/) (2025)
- [ReMindRAG: Low-Cost LLM-Guided Knowledge Graph Traversal for Efficient RAG](https://mlsanity.com/paper/2510.13193/) (2025)
- [An automatic patent literature retrieval system based on LLM-RAG](https://mlsanity.com/paper/2508.14064/) (2025)
- [Towards Context-Robust LLMs: A Gated Representation Fine-tuning Approach](https://mlsanity.com/paper/2502.14100/) (2025)
- [RAGVA: Engineering Retrieval Augmented Generation-based Virtual Assistants in Practice](https://mlsanity.com/paper/2502.14930/) (2025)
- [Hybrid-NL2SVA: Integrating RAG and Finetuning for LLM-based NL2SVA](https://mlsanity.com/paper/2506.21569/) (2025)
- [Parametric Retrieval-Augmented Generation using Latent Routing of LoRA Adapters](https://mlsanity.com/paper/2511.17044/) (2025)
- [SustainableQA: A Comprehensive Question Answering Dataset for Corporate Sustainability and EU Taxonomy Reporting](https://mlsanity.com/paper/2508.03000/) (2025)
- [Token-Level Precise Attack on RAG: Searching for the Best Alternatives to Mislead Generation](https://mlsanity.com/paper/2508.03110/) (2025)
- [Comparison of Text-Based and Image-Based Retrieval in Multimodal Retrieval Augmented Generation Large Language Model Systems](https://mlsanity.com/paper/2511.16654/) (2025)
- [From Topology to Retrieval: Decoding Embedding Spaces with Unified Signatures](https://mlsanity.com/paper/2511.22150/) (2025)
- [Towards Transparent RAG: Fostering Evidence Traceability in LLM Generation via Reinforcement Learning](https://mlsanity.com/paper/2505.13258/) (2025)
- [MAD-Fact: A Multi-Agent Debate Framework for Long-Form Factuality Evaluation in LLMs](https://mlsanity.com/paper/2510.22967/) (2025)
- [UniC-RAG: Universal Knowledge Corruption Attacks to Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.18652/) (2025)
- [KaPQA: Knowledge-Augmented Product Question-Answering](https://mlsanity.com/paper/2407.16073/) (2024)
- [KinyaColBERT: A Lexically Grounded Retrieval Model for Low-Resource Retrieval-Augmented Generation](https://mlsanity.com/paper/2507.03241/) (2025)
- [Adapting Large Language Models to Emerging Cybersecurity using Retrieval Augmented Generation](https://mlsanity.com/paper/2510.27080/) (2025)

## Abstract (arXiv)

The effectiveness of Large Language Models (LLMs) in generating accurate responses relies heavily on the quality of input provided, particularly when employing Retrieval Augmented Generation (RAG) techniques. RAG enhances LLMs by sourcing the most relevant text chunk(s) to base queries upon. Despite the significant advancements in LLMs' response quality in recent years, users may still encounter inaccuracies or irrelevant answers; these issues often stem from suboptimal text chunk retrieval by RAG rather than the inherent capabilities of LLMs. To augment the efficacy of LLMs, it is crucial to refine the RAG process. This paper explores the existing constraints of RAG pipelines and introduces methodologies for enhancing text retrieval. It delves into strategies such as sophisticated chunking techniques, query expansion, the incorporation of metadata annotations, the application of re-ranking algorithms, and the fine-tuning of embedding algorithms. Implementing these approaches can substantially improve the retrieval quality, thereby elevating the overall performance and reliability of LLMs in processing and responding to queries.