# Multi-Head RAG: Solving Multi-Aspect Problems with LLMs

arXiv: https://arxiv.org/abs/2406.05085 · HTML: https://mlsanity.com/paper/2406.05085/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 32 of its 41 references that are in the map.
- First posted on arXiv 7 Jun 2024.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval](https://mlsanity.com/paper/2401.18059/) (2024)
- [Ragas: Automated Evaluation of Retrieval Augmented Generation](https://mlsanity.com/paper/2309.15217/) (2023)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [Benchmarking Retrieval-Augmented Generation for Medicine](https://mlsanity.com/paper/2402.13178/) (2024)
- [Benchmarking Large Language Models in Retrieval-Augmented Generation](https://mlsanity.com/paper/2309.01431/) (2023)
- [Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models](https://mlsanity.com/paper/2311.09210/) (2023)
- [A Survey on Retrieval-Augmented Text Generation](https://mlsanity.com/paper/2202.01110/) (2022)
- [Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks](https://mlsanity.com/paper/2309.17410/) (2023)
- [Evaluation of Retrieval-Augmented Generation: A Survey](https://mlsanity.com/paper/2405.07437/) (2024)
- [On Protecting the Data Privacy of Large Language Models (LLMs): A Survey](https://mlsanity.com/paper/2403.05156/) (2024)
- [Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering](https://mlsanity.com/paper/2404.17723/) (2024)
- [CEDR: Contextualized Embeddings for Document Ranking](https://mlsanity.com/paper/1904.07094/) (2019)
- [Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipeline](https://mlsanity.com/paper/2101.08751/) (2021)
- [PARADE: Passage Representation Aggregation for Document Reranking](https://mlsanity.com/paper/2008.09093/) (2020)
- [RAG-Fusion: a New Take on Retrieval-Augmented Generation](https://mlsanity.com/paper/2402.03367/) (2024)
- [History, Development, and Principles of Large Language Models-An Introductory Survey](https://mlsanity.com/paper/2402.06853/) (2024)
- [HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responses](https://mlsanity.com/paper/2312.15883/) (2023)
- [In Defense of Cross-Encoders for Zero-Shot Retrieval](https://mlsanity.com/paper/2212.06121/) (2022)
- [Graph-Based Retriever Captures the Long Tail of Biomedical Knowledge](https://mlsanity.com/paper/2402.12352/) (2024)
- [Federated Recommendation via Hybrid Retrieval Augmented Generation](https://mlsanity.com/paper/2403.04256/) (2024)
- [Cross-Data Knowledge Graph Construction for LLM-enabled Educational Question-Answering System: A Case Study at HCMUT](https://mlsanity.com/paper/2404.09296/) (2024)

## Built on by (most cited first, 18 of 18)

- [When Search Engine Services meet Large Language Models: Visions and Challenges](https://mlsanity.com/paper/2407.00128/) (2024)
- [Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks](https://mlsanity.com/paper/2407.21059/) (2024)
- [Reasoning Language Models: A Blueprint](https://mlsanity.com/paper/2501.11223/) (2025)
- [DualRAG: A Dual-Process Approach to Integrate Reasoning and Retrieval for Multi-Hop Question Answering](https://mlsanity.com/paper/2504.18243/) (2025)
- [MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented Generation System](https://mlsanity.com/paper/2503.09600/) (2025)
- [RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Serving](https://mlsanity.com/paper/2503.14649/) (2025)
- [FinSage: A Multi-aspect RAG System for Financial Filings Question Answering](https://mlsanity.com/paper/2504.14493/) (2025)
- [Meta-Chunking: Learning Text Segmentation and Semantic Completion via Logical Perception](https://mlsanity.com/paper/2410.12788/) (2024)
- [SuperRAG: Beyond RAG with Layout-Aware Graph Modeling](https://mlsanity.com/paper/2503.04790/) (2025)
- [Hierarchical Lexical Graph for Enhanced Multi-Hop Retrieval](https://mlsanity.com/paper/2506.08074/) (2025)
- [Affordable AI Assistants with Knowledge Graph of Thoughts](https://mlsanity.com/paper/2504.02670/) (2025)
- [A Survey on Proactive Defense Strategies Against Misinformation in Large Language Models](https://mlsanity.com/paper/2507.05288/) (2025)
- [Securing RAG: A Risk Assessment and Mitigation Framework](https://mlsanity.com/paper/2505.08728/) (2025)
- [When Should Dense Retrievers Be Updated in Evolving Corpora? Detecting Out-of-Distribution Corpora Using GradNormIR](https://mlsanity.com/paper/2506.01877/) (2025)
- [SymbioticRAG: Enhancing Document Intelligence Through Human-LLM Symbiotic Collaboration](https://mlsanity.com/paper/2505.02418/) (2025)
- [Psychologically Enhanced AI Agents](https://mlsanity.com/paper/2509.04343/) (2025)
- [Think Parallax: Solving Multi-Hop Problems via Multi-View Knowledge-Graph-Based Retrieval-Augmented Generation](https://mlsanity.com/paper/2510.15552/) (2025)
- [Performance Foundations of Parallel & Distributed Reasoning Language Models](https://mlsanity.com/paper/2608.27046/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Generation (RAG) improves Large Language Models (LLMs) by retrieving supporting documents into the prompt, but existing methods do not explicitly target queries that require fetching multiple documents with substantially different content. Such multi-aspect queries are challenging because relevant documents can be far apart in embedding space, making joint retrieval difficult. We introduce Multi-Head RAG (MRAG), which addresses this gap with a simple yet powerful idea: using Transformer multi-head attention activations rather than the standard decoder-layer embedding, as retrieval keys. It leverages the observation that different heads capture different semantic aspects. This yields multi-aspect embeddings for both documents and queries, improving retrieval accuracy on complex queries. We show MRAG's design advantages over 18 RAG baselines, up to 20% higher retrieval success ratios for real-world use cases, and improved downstream LLM generation. MRAG integrates seamlessly with existing RAG frameworks and benchmarks.