RAG Deserves an Index: Why Ingest-Time Compilation Beats Query-Time Interpretation
On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval
Abstract (arXiv)
Nearly every retrieval-augmented question-answering system in production ships with a hidden interpreter: on each query a language model re-derives the meaning of raw corpus text and then throws that work away. Cheaper models do not close the gap: per-token prices have fallen by orders of magnitude while inference spend has risen, because context volume grows faster than prices fall. This is the modern equivalent of the full-table scan, and the remedy is the one databases found fifty years ago: do the expensive work once, at write time, into a maintained structure that makes reads cheap. A corpus whose read pattern is known before it ever meets a user can and should be indexed too. We call the paradigm ingest-time semantic compilation (ISC): compile a corpus's meaning into a queryable substrate with two coupled layers - incrementally maintained embeddings, and atomic claims whose provenance is validated at compile time - and treat that substrate as a first-class database object with its own DDL, maintenance contract, migration contract, and cost model. Two existence proofs support it. Substrate upkeep scales with change rather than corpus size: incremental updates run 33.7x cheaper than reconstruction while tracking it to floating-point precision. And on a held-out sample of 500 broadcast-interview transcripts, compiled claims as the retrieval payload win all 32 budget-by-model cells: 85.2% correct from roughly 2.2k reader tokens against 72.5% from 16.3k for the best chunk configuration anywhere. The only baseline that keeps pace is a contextualized-chunk pipeline with hybrid retrieval and reranking, statistically indistinguishable from compiled claims at roughly twenty-one times the query-path tokens - and it reaches that parity, we argue, precisely because it has itself begun to compile. We close with the systems agenda this opens, from compilation planners to read planning.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 11 of its 14 references that are in the map. First posted on arXiv 21 Aug 2026.
Builds on
11 of 14 references.
References · 11
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
- 2025Zep: A Temporal Knowledge Graph Architecture for Agent Memorycited by 372
- 2019Document Expansion by Query Predictioncited by 225
- 2021MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarizationcited by 104
- 2023Dense X Retrieval: What Retrieval Granularity Should We Use?cited by 88
- 2025HaluMem: Evaluating Hallucinations in Memory Systems of Agentscited by 51
- 2026Beyond the Context Window: A Cost-Performance Analysis of Fact-Based Memory vs. Long-Context LLMs for Persistent Agentscited by 13
- 2026Cost Scales with Change, Not Corpus Size: Incrementally Maintaining an Evolving Semantic Substratecited by 2
- 2026The Token Tax of Epistemic Accuracy: Comparing RAG and Long-Context Architectures for Document-Grounded Generative AI Applicationscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Ingest-Time Fact Compilation for Cost-Efficient and Reliable Question Answering over Revised Corpora
- 2026Forgotten History or Test-of-Time? Retrospect and Prospect on RAG from an IR Perspective
- 2026Semantically Similar, Logically Distinct: Diagnosing the Semantic-Answerability Gap in Table RAG
- 2026MemDelta: Controlled Baselines and Hidden Confounds in Agent Memory Evaluation
- 2025Reading Between the Timelines: RAG for Answering Diachronic Questions
- 2026Rethinking Retrieval-Augmentation as Synthesis: A Query-Aware Context Merging Approach
- 2026Can Small Language Models Use What They Retrieve? An Empirical Study of Retrieval Utilization Across Model Scale
- 2026QO-Bench: Diagnosing Query-Operator-Preserving Retrieval over Typed Event Tuples