Context-Efficient Retrieval with Factual Decomposition
Not on the map. Why?
Abstract (arXiv)
There has recently been considerable interest in incorporating information retrieval into large language models (LLMs). Retrieval from a dynamically expanding external corpus of text allows a model to incorporate current events and can be viewed as a form of episodic memory. Here we demonstrate that pre-processing the external corpus into semi-structured ''atomic facts'' makes retrieval more efficient. More specifically, we demonstrate that our particular form of atomic facts improves performance on various question answering tasks when the amount of retrieved text is limited. Limiting the amount of retrieval reduces the size of the context and improves inference efficiency.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 23 of its 41 references that are in the map. First posted on arXiv 25 Mar 2025.
Builds on
23 of 41 references.
References · 23
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
- 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
- 2021Improving language models by retrieving from trillions of tokenscited by 1,278
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
- 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
- 2023In-Context Retrieval-Augmented Language Modelscited by 694
- 2017The NarrativeQA Reading Comprehension Challengecited by 590
- 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
11 more
- 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
- 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
- 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
- 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
- 2023Unlimiformer: Long-Range Transformers with Unlimited Length Inputcited by 127
- 2024xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Tokencited by 123
- 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 104
- 2024CompAct: Compressing Retrieved Documents Actively for Question Answeringcited by 64
- 2024Molecular Facts: Desiderata for Decontextualization in LLM Fact Verificationcited by 42
- 2024Empowering Large Language Models to Set up a Knowledge Retrieval Indexer via Self-Learningcited by 13
- 2024LLM-Augmented Retrieval: Enhancing Retrieval Models Through Language Models and Doc-Level Embeddingcited by 6
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.