mlsanityData to 14 Oct 2026

LLMs Remember First, Forget Last: Dual-Process Interference in Large Language Models

arXiv 2603.00270 ↗First posted 27 Feb 2026cs.IR cs.AI cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Large language models can process millions of tokens, yet how they handle conflicting information within context remains poorly understood. From patient health logs tracking evolving vital signs to legal documents with superseding clauses, real-world applications routinely require models to retrieve specific values from streams of semantically similar, competing updates. We adapt classical interference paradigms from cognitive psychology to compare retroactive interference (RI; recalling initial values after updates) and proactive interference (PI; recalling recent values despite competing prior encodings) across 39 LLMs spanning 1B to 2.5T parameters. Every model exhibits the same pattern: PI causes substantially greater degradation than RI (d = 1.73), the opposite of the typical human finding where new information more readily disrupts old. Four lines of evidence indicate that RI and PI engage distinct mechanisms: model size predicts RI resistance but not PI; the two scores correlate only weakly; error analysis reveals qualitatively different failure profiles, with RI failures reflecting passive retrieval failure and PI failures reflecting active primacy intrusion; and hallucination rates remain below 1% in both conditions. An adaptive bootstrap design further shows that RI decays smoothly while PI collapses bimodally at high interference, consistent with gradual dilution versus winner-take-all attention dynamics. These findings characterize a systematic primacy bias in transformer attention, with implications for reliable deployment in settings where in-context information evolves over time.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 17 of its 32 references that are in the map. First posted on arXiv 27 Feb 2026.

Builds on

17 of 32 references.

References · 17

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  6. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  8. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  9. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  10. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  11. 2022Toy Models of Superpositioncited by 971
  12. 2017The NarrativeQA Reading Comprehension Challengecited by 597
5 more
  1. 2025NoLiMa: Long-Context Evaluation Beyond Literal Matchingcited by 126
  2. 2022Polysemanticity and Capacity in Neural Networkscited by 77
  3. 2025LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansioncited by 50
  4. 2025Unable to Forget: Proactive Interference Reveals Working Memory Limits in LLMs Beyond Context Lengthcited by 20
  5. 2025AbsenceBench: Language Models Can't Tell What's Missingcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.