# Financial Report Chunking for Effective Retrieval Augmented Generation

arXiv: https://arxiv.org/abs/2402.05131 · HTML: https://mlsanity.com/paper/2402.05131/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Placed in

- Retrieval-augmented generation / [Multimodal RAG](https://mlsanity.com/rag/multimodal-rag/)

## Builds on (most cited first, 28 of 28)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [FinanceBench: A New Benchmark for Financial Question Answering](https://mlsanity.com/paper/2311.11944/) (2023)
- [Challenges and Applications of Large Language Models](https://mlsanity.com/paper/2307.10169/) (2023)
- [FinGPT: Open-Source Financial Large Language Models](https://mlsanity.com/paper/2306.06031/) (2023)
- [ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering](https://mlsanity.com/paper/2210.03849/) (2022)
- [DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis](https://mlsanity.com/paper/2206.01062/) (2022)
- [Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Context](https://mlsanity.com/paper/2005.00589/) (2020)
- [Seven Failure Points When Engineering a Retrieval Augmented Generation System](https://mlsanity.com/paper/2401.05856/) (2024)
- [WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain](https://mlsanity.com/paper/2211.00083/) (2022)
- [Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?](https://mlsanity.com/paper/2309.07462/) (2023)
- [Cognitive Mirage: A Review of Hallucinations in Large Language Models](https://mlsanity.com/paper/2309.06794/) (2023)
- [Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models](https://mlsanity.com/paper/2306.12659/) (2023)
- [Assessing the Quality of Multiple-Choice Questions Using GPT-4 and Rule-Based Methods](https://mlsanity.com/paper/2307.08161/) (2023)
- [REFinD: Relation Extraction Financial Dataset](https://mlsanity.com/paper/2305.18322/) (2023)
- [ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?](https://mlsanity.com/paper/2311.16989/) (2023)
- [Zero-Shot Question Answering over Financial Documents using Large Language Models](https://mlsanity.com/paper/2311.14722/) (2023)
- [Context Tuning for Retrieval Augmented Generation](https://mlsanity.com/paper/2312.05708/) (2023)
- [Sentiment Analysis in Finance: From Transformers Back to eXplainable Lexicons (XLex)](https://mlsanity.com/paper/2306.03997/) (2023)
- [Conversational Factor Information Retrieval Model (ConFIRM)](https://mlsanity.com/paper/2310.13001/) (2023)

## Built on by (most cited first, 30 of 46)

- [HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction](https://mlsanity.com/paper/2408.04948/) (2024)
- [A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges](https://mlsanity.com/paper/2406.11903/) (2024)
- [Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely](https://mlsanity.com/paper/2409.14924/) (2024)
- [ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference](https://mlsanity.com/paper/2502.00299/) (2025)
- [Improving Retrieval for RAG based Question Answering Models on Financial Documents](https://mlsanity.com/paper/2404.07221/) (2024)
- [Ragnar"ok: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Track](https://mlsanity.com/paper/2406.16828/) (2024)
- [AI Hiring with LLMs: A Context-Aware and Explainable Multi-Agent Framework for Resume Screening](https://mlsanity.com/paper/2504.02870/) (2025)
- [Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2504.14891/) (2025)
- [FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.15800/) (2025)
- [CPA-RAG:Covert Poisoning Attacks on Retrieval-Augmented Generation in Large Language Models](https://mlsanity.com/paper/2505.19864/) (2025)
- [MarketSenseAI 2.0: Enhancing Stock Analysis through LLM Agents](https://mlsanity.com/paper/2502.00415/) (2025)
- [RAG Meets Temporal Graphs: Time-Sensitive Modeling and Retrieval for Evolving Knowledge](https://mlsanity.com/paper/2510.13590/) (2025)
- [Optimizing Retrieval Strategies for Financial Question Answering Documents in Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2503.15191/) (2025)
- [MultiFinRAG: An Optimized Multimodal Retrieval-Augmented Generation (RAG) Framework for Financial Question Answering](https://mlsanity.com/paper/2506.20821/) (2025)
- [FinSight: Towards Real-World Financial Deep Research](https://mlsanity.com/paper/2510.16844/) (2025)
- [Retrieval-Augmented Purifier for Robust LLM-Empowered Recommendation](https://mlsanity.com/paper/2504.02458/) (2025)
- [Chunk Twice, Embed Once: A Systematic Study of Segmentation and Representation Trade-offs in Chemistry-Aware Retrieval-Augmented Generation](https://mlsanity.com/paper/2506.17277/) (2025)
- [Enhancing Tourism Recommender Systems for Sustainable City Trips Using Retrieval-Augmented Generation](https://mlsanity.com/paper/2409.18003/) (2024)
- [Empowering Meta-Analysis: Leveraging Large Language Models for Scientific Synthesis](https://mlsanity.com/paper/2411.10878/) (2024)
- [REIS: A High-Performance and Energy-Efficient Retrieval System with In-Storage Processing](https://mlsanity.com/paper/2506.16444/) (2025)
- [Metadata-Driven Retrieval-Augmented Generation for Financial Question Answering](https://mlsanity.com/paper/2510.24402/) (2025)
- [Taxonomy of the Retrieval System Framework: Pitfalls and Paradigms](https://mlsanity.com/paper/2601.20131/) (2026)
- [A Scalable Data-Driven Framework for Systematic Analysis of SEC 10-K Filings Using Large Language Models](https://mlsanity.com/paper/2409.17581/) (2024)
- [LLMs for Supply Chain Management](https://mlsanity.com/paper/2505.18597/) (2025)
- [A Systematic Investigation of Document Chunking Strategies and Embedding Sensitivity](https://mlsanity.com/paper/2603.06976/) (2026)
- [Fine-Grained Privacy Extraction from Retrieval-Augmented Generation Systems via Knowledge Asymmetry Exploitation](https://mlsanity.com/paper/2507.23229/) (2025)
- [CelloAI: Leveraging Large Language Models for HPC Software Development in High Energy Physics](https://mlsanity.com/paper/2508.16713/) (2025)
- [An Agile Method for Implementing Retrieval Augmented Generation Tools in Industrial SMEs](https://mlsanity.com/paper/2508.21024/) (2025)
- [Knowledge-Grounded Agentic Large Language Models for Multi-Hazard Understanding from Reconnaissance Reports](https://mlsanity.com/paper/2511.14010/) (2025)
- [Legal-DC: Benchmarking Retrieval-Augmented Generation for Legal Documents](https://mlsanity.com/paper/2603.11772/) (2026)

## Abstract (arXiv)

Chunking information is a key step in Retrieval Augmented Generation (RAG). Current research primarily centers on paragraph-level chunking. This approach treats all texts as equal and neglects the information contained in the structure of documents. We propose an expanded approach to chunk documents by moving beyond mere paragraph-level chunking to chunk primary by structural element components of documents. Dissecting documents into these constituent elements creates a new way to chunk documents that yields the best chunk size without tuning. We introduce a novel framework that evaluates how chunking based on element types annotated by document understanding models contributes to the overall context and accuracy of the information retrieved. We also demonstrate how this approach impacts RAG assisted Question & Answer task performance. Our research includes a comprehensive analysis of various element types, their role in effective information retrieval, and the impact they have on the quality of RAG outputs. Findings support that element type based chunking largely improve RAG results on financial reporting. Through this research, we are also able to answer how to uncover highly accurate RAG.