# Structure-Aware Semantic Chunking with Title-Chain Prefixes: A 1600-Query Evaluation and the Measurement Trap in Text-Transform Ablations

arXiv: https://arxiv.org/abs/2608.00824 · HTML: https://mlsanity.com/paper/2608.00824/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 8 of its 12 references that are in the map.
- First posted on arXiv 1 Aug 2026.

## Placed in

- Retrieval-augmented generation / [Multimodal RAG](https://mlsanity.com/rag/multimodal-rag/)

## Builds on (most cited first, 8 of 8)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval](https://mlsanity.com/paper/2401.18059/) (2024)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [Rethinking Chunk Size For Long-Document Retrieval: A Multi-Dataset Analysis](https://mlsanity.com/paper/2505.21700/) (2025)
- [The Effect of Text Chunk Size on Retrieval-Augmented Generation Performance](https://mlsanity.com/paper/2607.24767/) (2026)

## Abstract (arXiv)

Chunking is the first and most consequential step in retrieval-augmented generation (RAG): every downstream retrieval decision inherits the chunk boundaries. We present a three-stage, chunk-side-only semantic chunking pipeline---header-split, semantic merge, and title-chain prefixing---that costs zero additional LLM calls: the title chain reuses the document's own header hierarchy instead of a generated summary. On a 1600-query stratified evaluation over a production Markdown knowledge base, the pipeline improves MRR@5 from 0.374 to 0.463 (+23.8%) on the full set and from 0.828 to 0.925 (+11.7%) on the answerable subset (n=563), with dual-annotator Cohen's kappa 0.45 (unweighted, 16,000 score pairs). We then report what we tried and what failed: three query-side or architecture-level follow-ups are design dead ends (unevaluated---no comparable run artifacts), one measured failure (prefix weight decay), and one protocol-level failure that is the paper's central methodological finding. In a same-pool prefix on/off ablation, dual-annotator agreement collapsed from kappa 0.45 to 0.04 under identical prompts---stripping the title-chain context strips the disambiguation signal annotators need to agree on relevance. This measurement trap invalidates a common evaluation practice in chunking research and motivates retrieval-time per-candidate prefix evaluation, the direction we recommend from all our evidence.