Inject or Navigate? Token-Efficient Retrieval for LLM Analysis of Transactional Legal Documents
Not on the map. Why?
Abstract (arXiv)
Answering questions over a set of transactional legal documents is most simply done by injecting the whole corpus into the LLM's context window on every query. That baseline maximises retrieval recall, but its token footprint scales with the corpus rather than the question, and long-context degradation scales with it. We report what it took to replace full-corpus injection in a legal-document analysis system, comparing it against two structured retrieval modes over our proprietary structure-aware chunking: embedding retrieval (NAVEMBED) and LLM navigation over a compact structured index (NAVINDEX). On a 20-question benchmark with verified ground-truth answers, a position-bias-controlled, reference-anchored pairwise judge scored semantic retrieval with reranking tied with injection on 16 of 18 document-bound questions (injection preferred on 2) while attending to 17.3x fewer input tokens (a general-text-embedding (GTE) configuration reaches 29.9x at a lower tie rate); both modes were judged tied on the 2 out-of-scope controls. NAVINDEX was judged tied on all 18 at a 1.61x smaller total token footprint, a ~56x smaller answering context, and 25% lower dollar cost. We derive a closed-form caching-crossover rule: cached injection is cheaper in dollars only while the corpus stays below roughly ten times the retrieval payload. Scope and uncertainty are quantified in Section 8.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 35 references that are in the map. First posted on arXiv 7 Jul 2026.
Builds on
18 of 35 references.
References · 18
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
- 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
- 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
- 2023Large Language Models are not Fair Evaluatorscited by 1,085
- 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
- 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
6 more
- 2019Passage Re-ranking with BERTcited by 645
- 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 532
- 2021CUAD: An Expert-Annotated NLP Dataset for Legal Contract Reviewcited by 236
- 2021ContractNLI: A Dataset for Document-level Natural Language Inference for Contractscited by 148
- 2025Reconstructing Context: Evaluating Advanced Chunking Strategies for Retrieval-Augmented Generationcited by 13
- 2026Evaluating Chunking Strategies For Retrieval-Augmented Generation in Oil and Gas Enterprise Documentscited by 1
Built on by
Not cited yet.