# M-RAG: Semantic Key-Value Indexing for Retrieval-Augmented Generation

arXiv: https://arxiv.org/abs/2603.26667 · HTML: https://mlsanity.com/paper/2603.26667/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 30 of its 34 references that are in the map.
- First posted on arXiv 6 Jan 2026.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 30 of 30)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://mlsanity.com/paper/2105.03011/) (2021)
- [A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2405.06211/) (2024)
- [LightRAG: Simple and Fast Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.05779/) (2024)
- [Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG](https://mlsanity.com/paper/2501.09136/) (2025)
- [Dense X Retrieval: What Retrieval Granularity Should We Use?](https://mlsanity.com/paper/2312.06648/) (2023)
- [Argumentation Mining in User-Generated Web Discourse](https://mlsanity.com/paper/1601.02403/) (2016)
- [Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models](https://mlsanity.com/paper/2310.04027/) (2023)
- [Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2309.17050/) (2023)
- [Grounding Language Model with Chunking-Free In-Context Retrieval](https://mlsanity.com/paper/2402.09760/) (2024)
- [MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented Generation System](https://mlsanity.com/paper/2503.09600/) (2025)
- [Stronger Baselines for Retrieval-Augmented Generation with Long-Context Language Models](https://mlsanity.com/paper/2506.03989/) (2025)
- [AGRAG: Advanced Graph-based Retrieval-Augmented Generation for LLMs](https://mlsanity.com/paper/2511.05549/) (2025)
- [HeteRAG: A Heterogeneous Retrieval-augmented Generation Framework with Decoupled Knowledge Representations](https://mlsanity.com/paper/2504.10529/) (2025)
- [Retrieval Augmented Question Answering: When Should LLMs Admit Ignorance?](https://mlsanity.com/paper/2512.23836/) (2025)
- [When Retrieval Succeeds and Fails: Rethinking Retrieval-Augmented Generation for LLMs](https://mlsanity.com/paper/2510.09106/) (2025)
- [HiFi-RAG: Hierarchical Content Filtering and Two-Pass Generation for Open-Domain RAG](https://mlsanity.com/paper/2512.22442/) (2025)

## Abstract (arXiv)

Retrieval-augmented generation (RAG) turns external documents into evidence for large language models. In practice, this is also a data access problem: a system must decide what to index, what to retrieve, and what evidence to place in the context under a token budget. Most RAG pipelines use text chunks for both lookup and generation. This couples two different objectives. Retrieval benefits from compact and discriminative records, while generation needs contextual and faithful evidence. As a result, small chunks may fragment answer-bearing information, whereas large chunks may introduce noise and waste the context budget. We propose M-RAG, a semantic key-value indexing layer for budget-constrained RAG query processing. M-RAG extracts meta-markers from complete documents, where each record contains a retrieval key, an information value, and provenance pointers. Online retrieval operates over the key field, which can be searched by dense vector retrieval or sparse lexical retrieval; the paired values are returned as generation payloads and assembled under the token budget. Provenance pointers further support coverage validation and position-aware context ordering. This design separates the physical index entry from the evidence payload without changing the underlying retriever or generator. Experiments on LongBench QA subtasks show that M-RAG achieves competitive or better accuracy than representative chunk-based baselines, especially under tight token budgets. Further analyses show high document coverage, stronger robustness under expanding candidate corpora, and lower online retrieval latency. These results suggest that semantic key-value indexing is a practical access method for RAG workloads.