mlsanityData to 14 Oct 2026

Temporal Context Reinstatement Drives Episodic-Like Order Memory in Long-Context Language Models

arXiv 2607.22575 ↗First posted 5 Jun 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Human episodic memory supports the retrieval of experiences that unfold over extended timescales, yet the computational mechanisms underlying this ability remain debated due to the limited mechanistic accessibility in long-term memory experiments in humans. Long-context LLMs may offer promising ways to reveal plausible computational mechanisms that drive this type of retrieval. Here, we investigate whether and how LLMs capture the core behavioral signatures of episodic memory via a temporal order memory task. Using a new dataset of human behavior based on memory of a full-length novel, we show that models exhibit the same characteristic distance effect observed in humans on this task. We next apply long-context mechanistic interpretability analyses to uncover how models solve this task, and find that model performance relies on a one-dimensional temporal code that is reinstated during retrieval by a single time-reinstatement attention head. These findings support temporal context reinstatement as an important mechanism for episodic-like temporal-order memory in LLMs, offering new insights into how temporal aspects of long-term episodic memory may be instantiated in both artificial and biological systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 48 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

16 of 48 references.

References · 16

  1. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  2. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  3. 2023The Impact of Positional Encoding on Length Generalization in Transformerscited by 327
  4. 2022Transformer Language Models without Positional Encodings Still Learn Positional Informationcited by 150
  5. 2019Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)cited by 128
  6. 2024Human-inspired Episodic Memory for Infinite Context LLMscited by 62
  7. 2024ICLR: In-Context Learning of Representationscited by 50
  8. 2024Representational Analysis of Binding in Language Modelscited by 28
  9. 2025Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddingscited by 26
  10. 2025Key-value memory in the braincited by 24
  11. 2023Latent Positional Information is in the Self-Attention Variance of Transformer Language Models Without Positional Embeddingscited by 23
  12. 2023Large language models can segment narrative events similarly to humanscited by 16
4 more
  1. 2024Linking In-context Learning in Transformers to Human Episodic Memorycited by 15
  2. 2014Spectral Ranking using Seriationcited by 9
  3. 2024Position Information Emerges in Causal Transformers Without Positional Encodings via Similarity of Nearby Embeddingscited by 8
  4. 2025Large Language Models as Model Organisms for Human Associative Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Position: Episodic Memory is the Missing Piece for Long-Term LLM Agents
  2. 2026RaMem: Contextual Reinstatement for Long-term Agentic Memory
  3. 2025Beyond Semantics: How Temporal Biases Shape Retrieval in Transformer and State-Space Models
  4. 2026Human-Like Lifelong Memory: A Neuroscience-Grounded Architecture for Infinite Interaction
  5. 2026From Observation to Intervention: Memory in Brains and Large Language Models
  6. 2025GENESIS: A Generative Model of Episodic-Semantic Interaction
  7. 2026Emerging Human-like Strategies for Semantic Memory Foraging in Large Language Models
  8. 2025Forgetting as a Feature: Cognitive Alignment of Large Language Models