Semantic Compression Trees: Multi-Resolution Knowledge Retrieval via Hierarchical Semantic Residuals
On the map: Retrieval-augmented generation → Generation and context integration
Abstract (arXiv)
Retrieval-augmented generation relies mostly on flat, fixed-granularity indexes: documents are cut into uniform chunks and retrieved by similarity, discarding the hierarchical structure of the source. We introduce Semantic Compression Trees (SCT), a hierarchical index in which each node stores only its semantic residual -- the information it adds beyond its parent -- and retrieval proceeds by progressive descent from the root, so that per-query cost is governed by tree depth rather than collection size. We evaluate on QASPER (50 papers, 173 questions) under two protocols differing only in whether the benchmark supplies the relevant document, with bootstrap confidence intervals and paired significance tests throughout. The results are mixed and we report them as such. When the document is given, SCT with a zero-LLM extractive compressor matches dense retrieval on answer quality (0.274 vs. 0.277 F1, p = 0.37) using 30% fewer context tokens and no LLM calls to build the index, and residual storage beats storing full summaries at each node (0.274 vs. 0.205, p < 0.001). Increasing the collection fifty-fold multiplies flat retrieval's per-query scoring work by 48.9x and SCT's by 6.4x. Progressive descent itself is not supported. Retrieving the same residuals without the tree performs identically when the document is given (p = 0.27), and descent is substantially worse when the system must select the document (0.122 vs. 0.165, p < 0.001). Routing accuracy localises the cause: descent selects the correct paper 20.2% of the time against 39.3% for flat retrieval, because that choice is made from the root residual, the most compressed node in the tree. We conclude that the residual representation is worth keeping and top-down routing is not.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 32 of its 41 references that are in the map. First posted on arXiv 21 Aug 2026.
Builds on
32 of 41 references.
Closest ancestors · 4
Same direction, newest first
- 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models23 May
- 2023Active Retrieval Augmented Generation11 May
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May
- 2020REALM: Retrieval-Augmented Language Model Pre-Training10 Feb
Other references · 28
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
- 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
- 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
- 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
- 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
- 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
- 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
16 more
- 2017The NarrativeQA Reading Comprehension Challengecited by 597
- 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 532
- 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
- 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
- 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
- 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
- 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 446
- 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
- 2022Dense Text Retrieval based on Pretrained Language Models: A Surveycited by 136
- 2024Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Modelscited by 73
- 2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directionscited by 70
- 2022An Exploration of Hierarchical Attention Transformers for Efficient Long Document Classificationcited by 38
- 2023Biomedical knowledge graph-optimized prompt generation for large language modelscited by 31
- 2024Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generationcited by 23
- 2025Reconstructing Context: Evaluating Advanced Chunking Strategies for Retrieval-Augmented Generationcited by 13
- 2025Hierarchical Semantic Retrieval with Cobwebcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025SARA: Selective and Adaptive Retrieval-augmented Generation with Context Compression
- 2026T-Retriever: Tree-based Hierarchical Retrieval Augmented Generation for Textual Graphs
- 2025CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning
- 2025Concept than Document: Context Compression via AMR-based Conceptual Entropy
- 2024Contextual Compression in Retrieval-Augmented Generation for Large Language Models: A Survey
- 2026Structure-Aware Semantic Chunking with Title-Chain Prefixes: A 1600-Query Evaluation and the Measurement Trap in Text-Transform Ablations
- 2025Knowledge Compression via Question Generation: Enhancing Multihop Document Retrieval without Fine-tuning
- 2025From Context to EDUs: Faithful and Structured Context Compression via Elementary Discourse Unit Decomposition