mlsanityData to 7 Oct 2026

In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss

arXiv 2402.10790 ↗First posted 16 Feb 2024cs.CL cs.AI cs.LGcited by 36

Not on the map. Why?

Abstract (arXiv)

This paper addresses the challenge of processing long documents using generative transformer models. To evaluate different approaches, we introduce BABILong, a new benchmark designed to assess model capabilities in extracting and processing distributed facts within extensive texts. Our evaluation, which includes benchmarks for GPT-4 and RAG, reveals that common methods are effective only for sequences up to 104 elements. In contrast, fine-tuning GPT-2 with recurrent memory augmentations enables it to handle tasks involving up to 11× 106 elements. This achievement marks a substantial leap, as it is by far the longest input processed by any neural network model to date, demonstrating a significant improvement in the processing capabilities for long sequences.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 31 of its 39 references that are in the map. First posted on arXiv 16 Feb 2024.

Builds on

31 of 39 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  4. 2020Longformer: The Long-Document Transformercited by 3,362
  5. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  7. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  8. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  9. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  10. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  11. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  12. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
19 more
  1. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
  2. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 696
  3. 2014Neural Turing Machinescited by 630
  4. 2023Adapting Language Models to Compress Contextscited by 350
  5. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
  6. 2015Towards AI-Complete Question Answering: A Set of Prerequisite Toy Taskscited by 309
  7. 2023L-Eval: Instituting Standardized Evaluation for Long Context Language Modelscited by 232
  8. 2022Memorizing Transformerscited by 219
  9. 2023LongNet: Scaling Transformers to 1,000,000,000 Tokenscited by 201
  10. 2022Recurrent Memory Transformercited by 197
  11. 2023ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understandingcited by 175
  12. 2022Block-Recurrent Transformerscited by 107
  13. 2024Long Context Compression with Activation Beaconcited by 101
  14. 2023Scaling Transformer to 1M tokens and beyond with RMTcited by 97
  15. 2020MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioningcited by 79
  16. 2020Memformer: A Memory-Augmented Transformer for Sequence Modelingcited by 63
  17. 2023Retrieval-Pretrained Transformer: Long-range Language Modeling with Self-retrievalcited by 20
  18. 2022Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learningcited by 13
  19. 2022Explain My Surprise: Learning Efficient Long-Term Memory by Predicting Uncertain Outcomescited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 22; 2025: 9; 2026: 5.

36 papers.

Sort:

By area:

  1. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  2. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  3. 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 219
  4. 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 163
  5. 2024Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumptioncited by 96
  6. 2024MileBench: Benchmarking MLLMs in Long Contextcited by 86
  7. 2024From Isolated Conversations to Hierarchical Schemas: Dynamic Tree Memory Representation for LLMscited by 77
  8. 2024RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrievalcited by 62
  9. 2024Needle In A Multimodal Haystackcited by 56
  10. 2024Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Modelscited by 53
  11. 2024SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attentioncited by 39
  12. 2024Needle in the Haystack for Memory Based Large Language Modelscited by 32
24 more
  1. 2025TRIX: A More Expressive Model for Zero-shot Domain Transfer in Knowledge Graphscited by 23
  2. 2024DetectiveQA: Evaluating Long-Context Reasoning on Detective Novelscited by 21
  3. 2024VideoLLaMB: Long Streaming Video Understanding with Recurrent Memory Bridgescited by 20
  4. 2024Associative Recurrent Memory Transformercited by 15
  5. 2024Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks?cited by 12
  6. 2025Context Is What You Need: The Maximum Effective Context Window for Real World Limits of LLMscited by 11
  7. 2025VTCBench: Can Vision-Language Models Understand Long Context with Vision-Text Compression?cited by 11
  8. 2024Bug In the Code Stack: Can LLMs Find Bugs in Large Python Code Stackscited by 6
  9. 2026ActMem: Bridging the Gap Between Memory Retrieval and Reasoning in LLM Agentscited by 6
  10. 2024Enhancing Supermarket Robot Interaction: A Multi-Level LLM Conversational Interface for Handling Diverse Customer Intentscited by 5
  11. 2026What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agentscited by 4
  12. 2025SolidMark: Evaluating Image Memorization in Generative Modelscited by 3
  13. 2025JobHop: A Large-Scale Dataset of Career Trajectoriescited by 3
  14. 2025NeMo: Needle in a Montage for Video-Language Understandingcited by 3
  15. 2024Assessing the quality of information extractioncited by 2
  16. 2026Core-based Hierarchies for Efficient GraphRAGcited by 2
  17. 2024Microstructures and Accuracy of Graph Recall by Large Language Modelscited by 1
  18. 2024Pay Attention to What Matterscited by 1
  19. 2025SCALAR: Scientific Citation-based Live Assessment of Long-context Academic Reasoningcited by 1
  20. 2026GraphRAG for Engineering Diagrams: ChatP&ID Enables LLM Interaction with P&IDscited by 1
  21. 2024Evaluating Long Range Dependency Handling in Code Generation LLMscited by 0
  22. 2024In-Context Ensemble Learning from Pseudo Labels Improves Video-Language Models for Low-Level Workflow Understandingcited by 0
  23. 2025Assessing Automated Fact-Checking for Medical LLM Responses with Knowledge Graphscited by 0
  24. 2026Efficient Retrieval-Augmented Generation via Token Co-occurrence Graphscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LOCOST: State-Space Models for Long Document Abstractive Summarization
  2. 2025LLM×MapReduce-V2: Entropy-Driven Convolutional Test-Time Scaling for Generating Long-Form Articles from Extremely Long Resources
  3. 2022Efficient Long Sequence Modeling via State Space Augmented Transformer
  4. 2024A Comprehensive Survey of Accelerated Generation Techniques in Large Language Models
  5. 2022Efficient Long Sequence Encoding via Synchronization
  6. 2024DeciMamba: Exploring the Length Extrapolation Potential of Mamba
  7. 2025Compress, Gather, and Recompute: REFORMing Long-Context Processing in Transformers
  8. 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack