mlsanityData to 7 Oct 2026

Lost in the Middle: How Language Models Use Long Contexts

arXiv 2307.03172 ↗First posted 6 Jul 2023cs.CLcited by 3,383

Not on the map. Why?

Cited by 744 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.

Key facts. Cited by 3,383 papers in the map (data through 7 Oct 2026). Builds on 42 of its 59 references that are in the map. First posted on arXiv 6 Jul 2023.

Builds on

42 of 59 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  6. 2020Longformer: The Long-Document Transformercited by 3,362
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  8. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  9. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  10. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  11. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  12. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
30 more
  1. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  2. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  3. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  4. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  5. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  6. 2020Efficient Transformers: A Surveycited by 784
  7. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 696
  8. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  9. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  10. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 495
  11. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  12. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 388
  13. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  14. 2021Random Feature Attentioncited by 350
  15. 2022UL2: Unifying Language Learning Paradigmscited by 292
  16. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 245
  17. 2020How Context Affects Language Models' Factual Predictionscited by 181
  18. 2023ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understandingcited by 175
  19. 2022SCROLLS: Standardized CompaRison Over Long Language Sequencescited by 170
  20. 2023MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformerscited by 164
  21. 2018Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Contextcited by 113
  22. 2021Do Long-Range Language Models Actually Use Long-Range Context?cited by 92
  23. 2020Shortformer: Better Language Modeling using Shorter Inputscited by 76
  24. 2022RankGen: Improving Text Generation with Large Ranking Modelscited by 72
  25. 2022Efficient Long-Text Understanding with Short-Text Modelscited by 71
  26. 2021What Context Features Can Transformer Language Models Use?cited by 49
  27. 2019Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Studycited by 47
  28. 2022The NLP Task Effectiveness of Long-Range Transformerscited by 22
  29. 2023Retrieval-Pretrained Transformer: Long-range Language Modeling with Self-retrievalcited by 20
  30. 2017Frustratingly Short Attention Spans in Neural Language Modelingcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 157; 2024: 680; 2025: 966; 2026: 1580.

3,383 papers.

Sort:

By area:

  1. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  2. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  5. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  6. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  7. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  8. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  9. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  10. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  11. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
  12. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
38 more
  1. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  2. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  3. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  4. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  5. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
  6. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
  7. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
  8. 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
  9. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 385
  10. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
  11. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 360
  12. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  13. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 343
  14. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  15. 2023Explainability for Large Language Models: A Surveycited by 331
  16. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 326
  17. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
  18. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  19. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
  20. 2024Titans: Learning to Memorize at Test Timecited by 303
  21. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 301
  22. 2024Long-context LLMs Struggle with Long In-context Learningcited by 285
  23. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  24. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 275
  25. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 270
  26. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  27. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 253
  28. 2023Challenges and Applications of Large Language Modelscited by 251
  29. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
  30. 2023L-Eval: Instituting Standardized Evaluation for Long Context Language Modelscited by 232
  31. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 230
  32. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agentscited by 226
  33. 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learningcited by 220
  34. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  35. 2024Repeat After Me: Transformers are Better than State Space Models at Copyingcited by 208
  36. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  37. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  38. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201

50 most cited of 3,383 All 3,383 (JSON)