Long Context Modeling with Ranked Memory-Augmented Retrieval
Not on the map. Why?
Abstract (arXiv)
Effective long-term memory management is crucial for language models handling extended contexts. We introduce the Enhanced Ranked Memory Augmented Retrieval (ERMAR) framework, which dynamically ranks memory entries based on relevance. Unlike prior models, ERMAR employs a novel relevance scoring mechanism and a pointwise re-ranking model for key-value embeddings, inspired by learning-to-rank techniques in information retrieval. By integrating historical usage patterns and adaptive retrieval, ERMAR achieves state-of-the-art results on standard benchmarks, demonstrating superior scalability and performance in long-context tasks.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 21 of its 30 references that are in the map. First posted on arXiv 19 Mar 2025.
Builds on
21 of 30 references.
References · 21
- 2017Attention Is All You Needcited by 59,932
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
- 2020Longformer: The Long-Document Transformercited by 3,362
- 2016Pointer Sentinel Mixture Modelscited by 2,992
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2020Big Bird: Transformers for Longer Sequencescited by 1,673
- 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
9 more
- 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
- 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
- 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
- 2014Neural Turing Machinescited by 630
- 2019Passage Re-ranking with BERTcited by 629
- 2024Data Engineering for Scaling Language Models to 128K Contextcited by 183
- 2023Focused Transformer: Contrastive Training for Context Scalingcited by 133
- 2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
- 2024Long-Context Language Modeling with Parallel Context Encodingcited by 84
Built on by
Citing papers per year: 2026: 2.
2 papers.