# ChunkRAG: Novel LLM-Chunk Filtering Method for RAG Systems

arXiv: https://arxiv.org/abs/2410.19572 · HTML: https://mlsanity.com/paper/2410.19572/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 25 Oct 2024.

## Placed in

- Retrieval-augmented generation / [Multimodal RAG](https://mlsanity.com/rag/multimodal-rag/)

## Builds on (most cited first, 16 of 16)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback](https://mlsanity.com/paper/2305.14387/) (2023)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [Making Retrieval-Augmented Language Models Robust to Irrelevant Context](https://mlsanity.com/paper/2310.01558/) (2023)
- [A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models](https://mlsanity.com/paper/2401.01313/) (2024)
- [Image Chat: Engaging Grounded Conversations](https://mlsanity.com/paper/1811.00945/) (2018)
- [RaFe: Ranking Feedback Improves Query Rewriting for RAG](https://mlsanity.com/paper/2405.14431/) (2024)
- [RA-ISF: Learning to Answer and Understand from Retrieval Augmentation via Iterative Self-Feedback](https://mlsanity.com/paper/2403.06840/) (2024)
- [Similarity is Not All You Need: Endowing Retrieval Augmented Generation with Multi Layered Thoughts](https://mlsanity.com/paper/2405.19893/) (2024)
- [Multi-Meta-RAG: Improving RAG for Multi-Hop Queries using Database Filtering with LLM-Extracted Metadata](https://mlsanity.com/paper/2406.13213/) (2024)
- [Reinforcement Retrieval Leveraging Fine-grained Feedback for Fact Checking News Claims with Black-Box LLM](https://mlsanity.com/paper/2404.17283/) (2024)

## Built on by (most cited first, 23 of 23)

- [VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videos](https://mlsanity.com/paper/2502.01549/) (2025)
- [A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions](https://mlsanity.com/paper/2507.18910/) (2025)
- [MiniRAG: Towards Extremely Simple Retrieval-Augmented Generation](https://mlsanity.com/paper/2501.06713/) (2025)
- [Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2504.14891/) (2025)
- [RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Serving](https://mlsanity.com/paper/2503.14649/) (2025)
- [FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.15800/) (2025)
- [Empowering GraphRAG with Knowledge Filtering and Integration](https://mlsanity.com/paper/2503.13804/) (2025)
- [Understanding DeepResearch via Reports](https://mlsanity.com/paper/2510.07861/) (2025)
- [Hybrid-NL2SVA: Integrating RAG and Finetuning for LLM-based NL2SVA](https://mlsanity.com/paper/2506.21569/) (2025)
- [PruneRAG: Confidence-Guided Query Decomposition Trees for Efficient Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.11024/) (2026)
- [Practical Code RAG at Scale: Task-Aware Retrieval Design Choices under Compute Budgets](https://mlsanity.com/paper/2510.20609/) (2025)
- [Agents-K1: Towards Agent-native Knowledge Orchestration](https://mlsanity.com/paper/2606.13669/) (2026)
- [RAGExplorer: A Visual Analytics System for the Comparative Diagnosis of RAG Systems](https://mlsanity.com/paper/2601.12991/) (2026)
- [Two-way Evidence self-Alignment based Dual-Gated Reasoning Enhancement](https://mlsanity.com/paper/2505.16806/) (2025)
- [LMAR: Language Model Augmented Retriever for Domain-specific Knowledge Indexing](https://mlsanity.com/paper/2508.05672/) (2025)
- [The Effect of Text Chunk Size on Retrieval-Augmented Generation Performance](https://mlsanity.com/paper/2607.24767/) (2026)
- [Harmonia: End-to-End RAG Serving Optimization](https://mlsanity.com/paper/2505.07833/) (2025)
- [Chunking Strategies for Multimodal AI Systems](https://mlsanity.com/paper/2512.00185/) (2025)
- [M'as contexto no es mejor. Paradoja de la diluci'on vectorial en RAG corporativos](https://mlsanity.com/paper/2601.08851/) (2025)
- [Rethinking Retrieval-Augmentation as Synthesis: A Query-Aware Context Merging Approach](https://mlsanity.com/paper/2603.20286/) (2026)
- [CHOP: Chunkwise Context-Preserving Framework for RAG on Multi Documents](https://mlsanity.com/paper/2604.15802/) (2026)
- [PolyUQuest: Verifiable Structure-Aware Web RAG over Heterogeneous Graphs](https://mlsanity.com/paper/2607.08269/) (2026)
- [RegExtract: A Benchmark for PFAS Regulatory Information Extraction with LLMs](https://mlsanity.com/paper/2610.15103/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Generation (RAG) systems using large language models (LLMs) often generate inaccurate responses due to the retrieval of irrelevant or loosely related information. Existing methods, which operate at the document level, fail to effectively filter out such content. We propose LLM-driven chunk filtering, ChunkRAG, a framework that enhances RAG systems by evaluating and filtering retrieved information at the chunk level. Our approach employs semantic chunking to divide documents into coherent sections and utilizes LLM-based relevance scoring to assess each chunk's alignment with the user's query. By filtering out less pertinent chunks before the generation phase, we significantly reduce hallucinations and improve factual accuracy. Experiments show that our method outperforms existing RAG models, achieving higher accuracy on tasks requiring precise information retrieval. This advancement enhances the reliability of RAG systems, making them particularly beneficial for applications like fact-checking and multi-hop reasoning.