mlsanityData to 7 Oct 2026

MDKeyChunker: Single-Call LLM Enrichment with Rolling Keys and Key-Based Restructuring for High-Accuracy RAG

arXiv 2603.23533 ↗First posted 8 Mar 2026cs.CL cs.AI cs.IRcited by 0

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Markdown carries structure a parser reads for free: headers, section paths, and block boundaries. Many RAG pipelines also spend LLM calls per chunk on generated metadata. We ask what one LLM call per chunk buys over that free structure. MDKeyChunker splits Markdown into header-led chunks without splitting any block; makes one LLM call per chunk for a title, summary, keywords, entities, questions, and a subtopic key, showing the model the keys already assigned in the document (a rolling key dictionary); and can merge same-key chunks. With qwen2.5:7b, we evaluate 79 Qasper questions over 30 papers and 73 FreshStack questions over 24 Laravel documentation files under BM25, two dense embedders, and hybrid fusion, following an analysis plan committed before results were computed. Evidence is matched only against source text, within a fixed token budget. Under hybrid retrieval, structural chunks beat 512-character windows on both datasets (Qasper +23.0 points, 95% CI [+12.8, +33.5]; FreshStack +5.1 [+1.4, +9.0]) and 256-token windows on Qasper (+12.7 [+5.3, +20.3]) but not on FreshStack (-2.6 [-6.4, +1.2]). Under the primary retrievers (hybrid, BM25), enrichment shows no planned-comparison difference from a free section-path prefix or from contextual retrieval; under hybrid retrieval the intervals exclude gains above 4.5 and 2.3 points on Qasper and 6.1 on FreshStack. Outside the planned comparisons, enrichment-style prefixes help BM25 on Qasper (exploratory) and mxbai on FreshStack (a secondary retriever). Rolling keys raise key reuse from 5.5% to 14.7%, but merging does not improve retrieval, and under BM25 on Qasper merging with rolling keys scores below merging without them (-6.0 [-13.1, -0.2]). Enrichment used about 1,000 input tokens per chunk; contextual retrieval 5,520 (Qasper) and 9,825 (FreshStack). The results of versions 1 and 2 are withdrawn.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 18 of its 37 references that are in the map. First posted on arXiv 8 Mar 2026.

Builds on

18 of 37 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Cross-Document Topic-Aligned Chunking for Retrieval-Augmented Generation8 Nov
  2. 2024Is Semantic Chunking Worth the Computational Cost?16 Oct
  3. 2024Meta-Chunking: Learning Text Segmentation and Semantic Completion via Logical Perception16 Oct
  4. 2024Financial Report Chunking for Effective Retrieval Augmented Generation5 Feb

Other references · 14

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  3. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  4. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
  5. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  6. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  7. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 273
  8. 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 255
  9. 2019Document Expansion by Query Predictioncited by 225
  10. 2023Dense X Retrieval: What Retrieval Granularity Should We Use?cited by 88
  11. 2024Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Modelscited by 70
  12. 2024LumberChunker: Long-Form Narrative Document Segmentationcited by 36
2 more
  1. 2025FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documentscited by 15
  2. 2025A Systematic Framework for Enterprise Knowledge Retrieval: Leveraging LLM-Generated Metadata to Enhance RAG Systemscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Structure-Aware Semantic Chunking with Title-Chain Prefixes: A 1600-Query Evaluation and the Measurement Trap in Text-Transform Ablations
  2. 2024ChunkRAG: Novel LLM-Chunk Filtering Method for RAG Systems
  3. 2026M-RAG: Semantic Key-Value Indexing for Retrieval-Augmented Generation
  4. 2025HiChunk: Evaluating and Enhancing Retrieval-Augmented Generation with Hierarchical Chunking
  5. 2026CHOP: Chunkwise Context-Preserving Framework for RAG on Multi Documents
  6. 2026Adaptive Chunking: Optimizing Chunking-Method Selection for RAG
  7. 2025Towards Global Retrieval Augmented Generation: A Benchmark for Corpus-Level Reasoning
  8. 2025MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented Generation System