mlsanityData to 14 Oct 2026

EASE-TTT: Evidence-Aligned Selective Test-Time Training for Long-Context Question Answering

arXiv 2606.06906 ↗First posted 5 Jun 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Long-context question answering (QA) remains challenging for smaller language models even when answer-bearing evidence is already present in the input. Existing within-context retrieval methods localize and expose candidate evidence chunks for the question, but they stop at input-level evidence exposure rather than adapting the query-side attention parameters that control how the model allocates attention over full-context positions. In contrast, lightweight test-time adaptation methods, such as query-only test-time training (qTTT), leave evidence localization unresolved because their generic span-level self-supervised objectives do not identify which context positions support the current answer. In this paper, we propose Evidence-Aligned SElective Test-Time Training (EASE-TTT), a within-context retrieval-augmented test-time training framework that converts selected evidence chunks into a soft attention supervision target over their token positions. Instead of replacing the full context with retrieved chunks, EASE-TTT uses the resulting attention target to guide query-side adaptation, with the adapted model generating the final answer from the original full context. Experiments on six LongBench QA tasks and three small decoder-only language models show that EASE-TTT achieves the strongest macro-average performance among full-context inference, retrieval-only baselines, and qTTT, supporting evidence-aligned test-time adaptation in long-context QA.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 44 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

32 of 44 references.

References · 32

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  3. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  4. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  5. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 532
  6. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 453
  7. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 348
  8. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 313
  9. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
  10. 2024Long-context LLMs Struggle with Long In-context Learningcited by 286
  11. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 256
  12. 2024A Human-Inspired Reading Agent with Gist Memory of Very Long Contextscited by 146
20 more
  1. 2025NoLiMa: Long-Context Evaluation Beyond Literal Matchingcited by 126
  2. 2024Make Your LLM Fully Utilize the Contextcited by 119
  3. 2024Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Modelscited by 73
  4. 2024CompAct: Compressing Retrieved Documents Actively for Question Answeringcited by 64
  5. 2024Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inferencecited by 53
  6. 2024LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answeringcited by 49
  7. 2024COME: Test-time adaption by Conservatively Minimizing Entropycited by 31
  8. 2024Grounding Language Model with Chunking-Free In-Context Retrievalcited by 30
  9. 2024Attend First, Consolidate Later: On the Importance of Attention in Different LLM Layerscited by 25
  10. 2024Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERTcited by 21
  11. 2025Efficient Context Selection for Long-Context QA: No Tuning, No Iteration, Just Adaptive-kcited by 21
  12. 2025U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystackcited by 20
  13. 2024ETHIC: Evaluating Large Language Models on Long-Context Tasks with High Information Coveragecited by 12
  14. 2025Eliciting In-context Retrieval and Reasoning for Long-context Large Language Modelscited by 11
  15. 2024Can't Remember Details in Long Documents? You Need Some R&Rcited by 10
  16. 2025Learning to Reason from Feedback at Test-Timecited by 10
  17. 2025Focus Directions Make Your Language Models Pay More Attention to Relevant Contextscited by 9
  18. 2023Drilling Down into the Discourse Structure with LLMs for Long Document Question Answeringcited by 8
  19. 2023Test-Time Self-Adaptive Small Language Models for Question Answeringcited by 7
  20. 2025Does RAG Really Perform Bad For Long-Context Processing?cited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Improving Context Fidelity via Native Retrieval-Augmented Reasoning
  2. 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA
  3. 2026Regime Boundary Alignment for Evidence-Gated Question Answering
  4. 2025T2: An Adaptive Test-Time Scaling Strategy for Contextual Question Answering
  5. 2026ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning
  6. 2024ALR2: A Retrieve-then-Reason Framework for Long-context Question Answering
  7. 2026EnComp: Lightweight Encoder-Only Context Compression for Retrieval-Augmented Question Answering
  8. 2025NeoQA: Evidence-based Question Answering with Generated News Events