mlsanityData to 14 Oct 2026

AttentionRetriever: Attention Layers are Secretly Long Document Retrievers

arXiv 2602.12278 ↗First posted 12 Feb 2026cs.IR cs.AIcited by 0

On the map: Retrieval-augmented generation → Generation and context integration

Abstract (arXiv)

Retrieval augmented generation (RAG) has been widely adopted to help Large Language Models (LLMs) to process tasks involving long documents. However, existing retrieval models are not designed for long document retrieval and fail to address several key challenges of long document retrieval, including context-awareness, causal dependence, and scope of retrieval. In this paper, we proposed AttentionRetriever, a novel long document retrieval model that leverages attention mechanism and entity-based retrieval to build context-aware embeddings for long document and determine the scope of retrieval. With extensive experiments, we found AttentionRetriever is able to outperform existing retrieval models on long document retrieval datasets by a large margin while remaining as efficient as dense retrieval models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 27 of its 42 references that are in the map. First posted on arXiv 12 Feb 2026.

Builds on

27 of 42 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May

Other references · 26

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  7. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  8. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  9. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  10. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  11. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
  12. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
14 more
  1. 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
  2. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
  3. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 313
  4. 2019Analyzing the Structure of Attention in a Transformer Language Modelcited by 243
  5. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 236
  6. 2024Generative Representational Instruction Tuningcited by 233
  7. 2024Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Modelscited by 73
  8. 2024A Controlled Study on Long Context Extension and Generalization in LLMscited by 26
  9. 2025LongRoPE2: Near-Lossless LLM Context Window Scalingcited by 24
  10. 2024ReAttention: Training-Free Infinite Context with Finite Attention Scopecited by 13
  11. 2025Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddingscited by 13
  12. 2023DAPR: A Benchmark on Document-Aware Passage Retrievalcited by 9
  13. 2025Infinite Retrieval: Attention Enhanced LLMs in Long-Context Processingcited by 7
  14. 2025Single-Pass Document Scanning for Question Answeringcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ELITE: Embedding-Less retrieval with Iterative Text Exploration
  2. 2025AttentionRAG: Attention-Guided Context Pruning in Retrieval-Augmented Generation
  3. 2026SproutRAG: Attention-Guided Tree Search with Progressive Embeddings for Long-Document RAG
  4. 2025ParetoRAG: Leveraging Sentence-Context Attention for Robust and Efficient Retrieval-Augmented Generation
  5. 2024Unveiling the Magic: Investigating Attention Distillation in Retrieval-augmented Generation
  6. 2025SitEmb-v1.5: Improved Context-Aware Dense Retrieval for Semantic Association and Long Story Comprehension
  7. 2025Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-ranking
  8. 2025Beyond RAG vs. Long-Context: Learning Distraction-Aware Retrieval for Efficient Knowledge Grounding