mlsanityData to 14 Oct 2026

Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generation

arXiv 2406.00456 ↗First posted 1 Jun 2024cs.LG cs.AI cs.CLcited by 23

On the map: Retrieval-augmented generation → Knowledge-graph and structured-knowledge RAG

Abstract (arXiv)

Integrating information from various reference databases is a major challenge for Retrieval-Augmented Generation (RAG) systems because each knowledge source adopts a unique data structure and follows different conventions. Retrieving from multiple knowledge sources with one fixed strategy usually leads to under-exploitation of information. To mitigate this drawback, inspired by Mix-of-Expert, we introduce Mix-of-Granularity (MoG), a method that dynamically determines the optimal granularity of a knowledge source based on input queries using a router. The router is efficiently trained with a newly proposed loss function employing soft labels. We further extend MoG to MoG-Graph (MoGG), where reference documents are pre-processed as graphs, enabling the retrieval of distantly situated snippets. Experiments demonstrate that MoG and MoGG effectively predict optimal granularity levels, significantly enhancing the performance of the RAG system in downstream tasks. The code of both MoG and MoGG are released in https://github.com/ZGChung/Mix-of-Granularity.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Knowledge-graph and structured-knowledge RAG, one of 15 directions in Retrieval-augmented generation. Builds on 20 of its 40 references that are in the map. First posted on arXiv 1 Jun 2024.

Builds on

20 of 40 references.

References · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2017Billion-scale similarity search with GPUscited by 2,497
  7. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  8. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  9. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  10. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  11. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
  12. 2018GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Modelscited by 413
8 more
  1. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 318
  2. 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactionscited by 206
  3. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 185
  4. 2023Graph Mixture of Experts: Learning on Large-Scale Graphs with Explicit Diversity Modelingcited by 64
  5. 2023Knowledge Graph-Augmented Language Models for Knowledge-Grounded Dialogue Generationcited by 54
  6. 2020CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Trainingcited by 38
  7. 2018Medical Exam Question Answering with Large-scale Reading Comprehensioncited by 29
  8. 2022EASE: Entity-Aware Contrastive Learning of Sentence Embeddingcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 13; 2026: 6.

23 papers.

Sort:

By area:

  1. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 88
  2. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 66
  3. 2025UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularitiescited by 20
  4. 2025PIKE-RAG: sPecIalized KnowledgE and Rationale Augmented Generationcited by 18
  5. 2025Rethinking Chunk Size For Long-Document Retrieval: A Multi-Dataset Analysiscited by 17
  6. 2025FinGEAR: Financial Mapping-Guided Enhanced Answer Retrievalcited by 5
  7. 2024RAG Playground: A Framework for Systematic Evaluation of Retrieval Strategies and Prompt Engineering in RAG Systemscited by 4
  8. 2025A Survey of LLM × DATAcited by 4
  9. 2025SemRAG: Semantic Knowledge-Augmented RAG for Improved Question-Answeringcited by 4
  10. 2025Dynamic Context Adaptation for Consistent Role-Playing Agents with Retrieval-Augmented Generationscited by 4
  11. 2026Hippocampus: An Efficient and Scalable Memory Module for Agentic AIcited by 3
  12. 2025Comparing Knowledge Injection Methods for LLMs in a Low-Resource Regimecited by 2
11 more
  1. 2026Are Finer Citations Always Better? Rethinking Granularity for Attributed Generationcited by 2
  2. 2024Advanced ingestion process powered by LLM parsing for RAG systemcited by 1
  3. 2025Smooth Reading: Bridging the Gap of Recurrent LLM to Self-Attention LLM on Long-Context Taskscited by 1
  4. 2026Failure is Feedback: History-Aware Backtracking for Agentic Traversal in Multimodal Graphscited by 1
  5. 2024PersianRAG: A Retrieval-Augmented Generation System for Persian Languagecited by 0
  6. 2025FreeChunker: A Cross-Granularity Chunking Frameworkcited by 0
  7. 2025Chunking Strategies for Multimodal AI Systemscited by 0
  8. 2025Breaking It Down: Domain-Aware Semantic Segmentation for Retrieval Augmented Generationcited by 0
  9. 2026TopoChunker: Topology-Aware Agentic Document Chunking Frameworkcited by 0
  10. 2026LLM-Based Intelligent Notification Composition: From Static Personalization to Context-Aware Persuasive Messagingcited by 0
  11. 2026Semantic Compression Trees: Multi-Resolution Knowledge Retrieval via Hierarchical Semantic Residualscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Multi-Source Knowledge Pruning for Retrieval-Augmented Generation: A Benchmark and Empirical Study
  2. 2026FlexStructRAG: Flexible Structure-Aware Multi-Granular Relational Retrieval for RAG
  3. 2024textttMixGR: Enhancing Retriever Generalization for Scientific Domain through Complementary Granularity
  4. 2024CoFE-RAG: A Comprehensive Full-chain Evaluation Framework for Retrieval-Augmented Generation with Enhanced Data Diversity
  5. 2026DA-RAG: Dynamic Attributed Community Search for Retrieval-Augmented Generation
  6. 2026Uncertainty-Aware Hybrid Retrieval for Long-Document RAG
  7. 2025HeteRAG: A Heterogeneous Retrieval-augmented Generation Framework with Decoupled Knowledge Representations
  8. 2025Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Survey