mlsanityData to 7 Oct 2026

MegaMem: A Retrieval Solution for Ultra-Large Context Windows

arXiv 2608.22137 ↗First posted 22 Aug 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Modern language models and agents increasingly require persistent memory for complete codebases, long interaction histories, and heterogeneous enterprise records. The key challenge is to keep hundreds of millions of tokens searchable while passing only bounded source evidence to the answer model. We introduce MegaMem, a source-resolved dual-view retrieval system that separates semantic access from generation evidence. Distilled records and detailed evidence are searched with original and transformed queries; every distilled hit resolves to an immutable source ID before reciprocal-rank fusion, deduplication, and cross-encoder reranking; and only the highest-ranked detailed evidence within a fixed budget supports generation. Post-answer attribution then identifies which loaded sources support the fixed answer. We evaluate MegaMem on EnterpriseRAG-Bench, which contains more than 500,000 heterogeneous enterprise documents and approximately 650M tokens. MegaMem improves Overall from 68.22 to 82.26 and reaches 86.50 Correctness. These results show that MegaMem supports ultra-large persistent memory while preserving strong answer accuracy under a bounded generation context. By separating searchable memory scale from answer-context size, MegaMem provides a practical path toward accurate retrieval over memories ranging from hundreds of millions to one billion tokens. Our code is available at https://github.com/ xfab-xinyuansong/MegaMem.git.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 35 of its 51 references that are in the map. First posted on arXiv 22 Aug 2026.

Builds on

35 of 51 references.

References · 35

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  6. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  7. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  8. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  9. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  10. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  11. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  12. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
23 more
  1. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
  2. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  3. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  4. 2019Passage Re-ranking with BERTcited by 629
  5. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  6. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  7. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  8. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 360
  9. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  10. 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 271
  11. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
  12. 2020Generation-Augmented Retrieval for Open-domain Question Answeringcited by 186
  13. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 126
  14. 2024MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentationcited by 118
  15. 2025Context Length Alone Hurts LLM Performance Despite Perfect Retrievalcited by 118
  16. 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 104
  17. 2024In Defense of RAG in the Era of Long-Context Language Modelscited by 52
  18. 2024RAGGED: Towards Informed Design of Scalable and Stable RAG Systemscited by 24
  19. 2025Benchmarking Deep Search over Heterogeneous Enterprise Datacited by 23
  20. 2023Worst-case Performance of Popular Approximate Nearest Neighbor Search Implementations: Guarantees and Limitationscited by 20
  21. 2025More Documents, Same Length: Isolating the Challenge of Multiple Documents in RAGcited by 16
  22. 2026EnterpriseRAG-Bench: A RAG Benchmark for Company Internal Knowledgecited by 7
  23. 2025Exp4Fuse: A Rank Fusion Framework for Enhanced Sparse Retrieval using Large Language Model-based Query Expansioncited by 6

Built on by

Not cited yet.