mlsanityData to 7 Oct 2026

Is Semantic Chunking Worth the Computational Cost?

arXiv 2410.13070 ↗First posted 16 Oct 2024cs.CL cs.IRcited by 43

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Recent advances in Retrieval-Augmented Generation (RAG) systems have popularized semantic chunking, which aims to improve retrieval performance by dividing documents into semantically coherent segments. Despite its growing adoption, the actual benefits over simpler fixed-size chunking, where documents are split into consecutive, fixed-size segments, remain unclear. This study systematically evaluates the effectiveness of semantic chunking using three common retrieval-related tasks: document retrieval, evidence retrieval, and retrieval-based answer generation. The results show that the computational costs associated with semantic chunking are not justified by consistent performance gains. These findings challenge the previous assumptions about semantic chunking and highlight the need for more efficient chunking strategies in RAG systems.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 17 of its 33 references that are in the map. First posted on arXiv 16 Oct 2024.

Builds on

17 of 33 references.

References · 17

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  5. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  6. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  7. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  8. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  9. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  10. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  11. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 426
  12. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 180
5 more
  1. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systemscited by 115
  2. 2023DelucionQA: Detecting Hallucinations in Domain-specific Question Answeringcited by 43
  3. 2024The Chronicles of RAG: The Retriever, the Chunk and the Generatorcited by 34
  4. 2021ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answerscited by 29
  5. 2024From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queriescited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 18; 2026: 24.

43 papers.

Sort:

By area:

  1. 2025LightMem: Lightweight and Efficient Memory-Augmented Generationcited by 163
  2. 2025MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented Generation Systemcited by 19
  3. 2025Towards Reliable Retrieval in RAG Systems for Large Legal Datasetscited by 12
  4. 2025Classifying and Addressing the Diversity of Errors in Retrieval-Augmented Generation Systemscited by 9
  5. 2026MultiDocFusion: Hierarchical and Multimodal Chunking Pipeline for Enhanced RAG on Long Industrial Documentscited by 6
  6. 2026Chain-of-Memory: Lightweight Memory Construction with Dynamic Evolution for LLM Agentscited by 5
  7. 2025SafeRBench: Dissecting the Reasoning Safety of Large Language Modelscited by 4
  8. 2026Taxonomy of the Retrieval System Framework: Pitfalls and Paradigmscited by 4
  9. 2025SemRAG: Semantic Knowledge-Augmented RAG for Improved Question-Answeringcited by 3
  10. 2025Practical Code RAG at Scale: Task-Aware Retrieval Design Choices under Compute Budgetscited by 3
  11. 2026A Systematic Investigation of Document Chunking Strategies and Embedding Sensitivitycited by 3
  12. 2024TOBUGraph: Knowledge Graph-Based Retrieval for Enhanced LLM Performance Beyond RAGcited by 2
31 more
  1. 2025Multi-Agent System for AI-Assisted Extraction of Narrative Arcs in TV Seriescited by 2
  2. 2025Can LLMs Replace Humans During Code Chunking?cited by 2
  3. 2025An Agile Method for Implementing Retrieval Augmented Generation Tools in Industrial SMEscited by 2
  4. 2026Paper Circle: An Open-source Multi-agent Research Discovery and Analysis Frameworkcited by 2
  5. 2026HiKEY: Hierarchical Multimodal Retrieval for Open-Domain Document Question Answeringcited by 2
  6. 2025Beyond Text: Characterizing Domain Expert Needs in Document Researchcited by 1
  7. 2025Less Context, Same Performance: A RAG Framework for Resource-Efficient LLM-Based Clinical NLPcited by 1
  8. 2025Knowledge Compression via Question Generation: Enhancing Multihop Document Retrieval without Fine-tuningcited by 1
  9. 2025MoM: Mixtures of Scenario-Aware Document Memories for Retrieval-Augmented Generation Systemscited by 1
  10. 2026Disco-RAG: Discourse-Aware Retrieval-Augmented Generationcited by 1
  11. 2026Semantic Chunking and the Entropy of Natural Languagecited by 1
  12. 2026Evaluating Chunking Strategies For Retrieval-Augmented Generation in Oil and Gas Enterprise Documentscited by 1
  13. 2026Adaptive Chunking: Optimizing Chunking-Method Selection for RAGcited by 1
  14. 2026Chunking Methods on Retrieval-Augmented Generation - Effectiveness Evaluation Against Computational Cost and Limitationscited by 1
  15. 2025Embedding-Based Context-Aware Rerankercited by 0
  16. 2025FreeChunker: A Cross-Granularity Chunking Frameworkcited by 0
  17. 2025Breaking It Down: Domain-Aware Semantic Segmentation for Retrieval Augmented Generationcited by 0
  18. 2025Retrieval-Augmented Generation Assistant for Anatomical Pathology Laboratoriescited by 0
  19. 2026MDKeyChunker: Single-Call LLM Enrichment with Rolling Keys and Key-Based Restructuring for High-Accuracy RAGcited by 0
  20. 2026AnchorMem: Anchored Facts with Associative Contexts for Building Memory in Large Language Modelscited by 0
  21. 2026From Detection to Response: A Deep Learning and Retrieval-Augmented Generation Framework for Network Intrusion Mitigationcited by 0
  22. 2026BLAgent: Agentic RAG for File-Level Bug Localizationcited by 0
  23. 2026M3DocDep: Multi-modal, Multi-page, Multi-document Dependency Chunking with Large Vision-Language Modelscited by 0
  24. 2026Evaluation of Chunking Strategies for Effective Text Embedding in Low-Resource Language on Agricultural Documentscited by 0
  25. 2026SEEK: Semantic Evidence Extraction via Adaptive ChunKing for Multilingual Fact-Checkingcited by 0
  26. 2026RAGe: A Retrieval-Augmented Generation Evaluation Frameworkcited by 0
  27. 2026Efficient RAG with Intent-Aware Retrieval and Semantics-Preserving Chunkingcited by 0
  28. 2026Storyline Trees: Hierarchical Representations for Long-Form Narrativescited by 0
  29. 2026GRASP: GRanularity-Aware Search Policy for Agentic RAGcited by 0
  30. 2026Cross-Attention Calibrated Deduplication for Retrieval-Augmented Generation Systemcited by 0
  31. 2026Studying quantization trade-offs for efficient inference deployment in machine translationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A New HOPE: Domain-agnostic Automatic Evaluation of Text Chunking
  2. 2026Evaluating Chunking Strategies for Retrieval-Augmented Generation on Academic Texts
  3. 2026Query-Adaptive Semantic Chunking for Retrieval-Augmented Generation: A Dynamic Strategy with Contextual Window Expansion
  4. 2024Financial Report Chunking for Effective Retrieval Augmented Generation
  5. 2026Web Retrieval-Aware Chunking (W-RAC) for Efficient and Cost-Effective Retrieval-Augmented Generation Systems
  6. 2026A Systematic Analysis of Chunking Strategies for Reliable Question Answering
  7. 2025HiChunk: Evaluating and Enhancing Retrieval-Augmented Generation with Hierarchical Chunking
  8. 2026The Effect of Text Chunk Size on Retrieval-Augmented Generation Performance