mlsanityData to 7 Oct 2026

Retrieval meets Long Context Large Language Models

arXiv 2310.03025 ↗First posted 4 Oct 2023cs.CL cs.AI cs.IRcited by 114

Not on the map. Why?

Cited by 36 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Extending the context window of large language models (LLMs) is getting popular recently, while the solution of augmenting LLMs with retrieval has existed for years. The natural questions are: i) Retrieval-augmentation versus long context window, which one is better for downstream tasks? ii) Can both methods be combined to get the best of both worlds? In this work, we answer these questions by studying both solutions using two state-of-the-art pretrained LLMs, i.e., a proprietary 43B GPT and Llama2-70B. Perhaps surprisingly, we find that LLM with 4K context window using simple retrieval-augmentation at generation can achieve comparable performance to finetuned LLM with 16K context window via positional interpolation on long context tasks, while taking much less computation. More importantly, we demonstrate that retrieval can significantly improve the performance of LLMs regardless of their extended context window sizes. Our best model, retrieval-augmented Llama2-70B with 32K context window, outperforms GPT-3.5-turbo-16k and Davinci003 in terms of average score on nine long context tasks including question answering, query-based summarization, and in-context few-shot learning tasks. It also outperforms its non-retrieval Llama2-70B-32k baseline by a margin, while being much faster at generation. Our study provides general insights on the choice of retrieval-augmentation versus long context extension of LLM for practitioners.

Key facts. Cited by 114 papers in the map (data through 7 Oct 2026). Builds on 65 of its 76 references that are in the map. First posted on arXiv 4 Oct 2023.

Builds on

65 of 76 references.

References · 65

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  10. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  11. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  12. 2020Longformer: The Long-Document Transformercited by 3,362
38 more
  1. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  2. 2022Emergent Abilities of Large Language Modelscited by 2,483
  3. 2017Billion-scale similarity search with GPUscited by 2,477
  4. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  5. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  6. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  7. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  8. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  9. 2020Reformer: The Efficient Transformercited by 1,658
  10. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  11. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  12. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  13. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  14. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  15. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  16. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  17. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  18. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  19. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  20. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  21. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  22. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  23. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  24. 2020Efficient Transformers: A Surveycited by 784
  25. 2018Image Transformercited by 770
  26. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  27. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  28. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
  29. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  30. 2019ELI5: Long Form Question Answeringcited by 567
  31. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  32. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 469
  33. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 468
  34. 2021QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarizationcited by 363
  35. 2019Axial Attention in Multidimensional Transformerscited by 350
  36. 2018Generating Wikipedia by Summarizing Long Sequencescited by 333
  37. 2022UL2: Unifying Language Learning Paradigmscited by 292
  38. 2020Sparse Sinkhorn Attentioncited by 250

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 56; 2025: 29; 2026: 25.

114 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  3. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  4. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 235
  5. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  6. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201
  7. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  8. 2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memorycited by 192
  9. 2023Efficient Large Language Models: A Surveycited by 183
  10. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAGcited by 122
  11. 2025FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inferencecited by 121
  12. 2024Make Your LLM Fully Utilize the Contextcited by 118
38 more
  1. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  2. 2024Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generationcited by 99
  3. 2024MARG: Multi-Agent Review Generation for Scientific Paperscited by 98
  4. 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAGcited by 90
  5. 2024Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generationcited by 78
  6. 2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systemscited by 76
  7. 2025ReasonIR: Training Retrievers for Reasoning Taskscited by 76
  8. 2024User-LLM: Efficient LLM Contextualization with User Embeddingscited by 74
  9. 2024Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encodingcited by 72
  10. 2023Thread of Thought Unraveling Chaotic Contextscited by 71
  11. 2024Inference Scaling for Long-Context Retrieval Augmented Generationcited by 65
  12. 2026Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontierscited by 60
  13. 2024Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Modelscited by 57
  14. 2024Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmarkcited by 53
  15. 2024ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilitiescited by 52
  16. 2025LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansioncited by 47
  17. 2024UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysiscited by 40
  18. 2024The Chronicles of RAG: The Retriever, the Chunk and the Generatorcited by 34
  19. 2024RAR-b: Reasoning as Retrieval Benchmarkcited by 33
  20. 2024NovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokenscited by 31
  21. 2024LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scalecited by 30
  22. 2025LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs -- No Silver Bullet for LC or RAG Routingcited by 29
  23. 2024V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encodingcited by 28
  24. 2024RAP: Retrieval-Augmented Personalization for Multimodal Large Language Modelscited by 24
  25. 2025CAM: A Constructivist View of Agentic Memory for LLM-Based Reading Comprehensioncited by 23
  26. 2024Are Long-LLMs A Necessity For Long-Context Tasks?cited by 22
  27. 2024DetectiveQA: Evaluating Long-Context Reasoning on Detective Novelscited by 21
  28. 2025Vision-centric Token Compression in Large Language Modelcited by 21
  29. 2025Toward Multi-Session Personalized Conversation: A Large-Scale Dataset and Hierarchical Tree Framework for Implicit Reasoningcited by 21
  30. 2024From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Datacited by 18
  31. 2024Sibyl: Simple yet Effective Agent Framework for Complex Real-world Reasoningcited by 17
  32. 2024Compressing Long Context for Enhancing RAG with AMR-based Concept Distillationcited by 16
  33. 2023ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?cited by 15
  34. 2024Losing Visual Needles in Image Haystacks: Vision Language Models are Easily Distracted in Short and Long Contextscited by 14
  35. 2024UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generationcited by 14
  36. 2024Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks?cited by 12
  37. 2025RARE: Retrieval-Augmented Reasoning Modelingcited by 11
  38. 2026Coding Agents are Effective Long-Context Processorscited by 11

50 most cited of 114 All 114 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Infinite Retrieval: Attention Enhanced LLMs in Long-Context Processing
  2. 2024Evaluating Multilingual Long-Context Models for Retrieval and Reasoning
  3. 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA
  4. 2024Enhancing Long Context Performance in LLMs Through Inner Loop Query Mechanism
  5. 2024An Effective Framework to Help Large Language Models Handle Numeric-involved Long-context Tasks
  6. 2024Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach
  7. 2024MemLong: Memory-Augmented Retrieval for Long Text Modeling
  8. 2024LongEmbed: Extending Embedding Models for Long Context Retrieval