# Temporal Context Reinstatement Drives Episodic-Like Order Memory in Long-Context Language Models

arXiv: https://arxiv.org/abs/2607.22575 · HTML: https://mlsanity.com/paper/2607.22575/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 48 references that are in the map.
- First posted on arXiv 5 Jun 2026.

## Builds on (most cited first, 16 of 16)

- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [The Impact of Positional Encoding on Length Generalization in Transformers](https://mlsanity.com/paper/2305.19466/) (2023)
- [Transformer Language Models without Positional Encodings Still Learn Positional Information](https://mlsanity.com/paper/2203.16634/) (2022)
- [Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)](https://mlsanity.com/paper/1905.11833/) (2019)
- [Human-inspired Episodic Memory for Infinite Context LLMs](https://mlsanity.com/paper/2407.09450/) (2024)
- [ICLR: In-Context Learning of Representations](https://mlsanity.com/paper/2501.00070/) (2024)
- [Representational Analysis of Binding in Language Models](https://mlsanity.com/paper/2409.05448/) (2024)
- [Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings](https://mlsanity.com/paper/2512.12167/) (2025)
- [Key-value memory in the brain](https://mlsanity.com/paper/2501.02950/) (2025)
- [Latent Positional Information is in the Self-Attention Variance of Transformer Language Models Without Positional Embeddings](https://mlsanity.com/paper/2305.13571/) (2023)
- [Large language models can segment narrative events similarly to humans](https://mlsanity.com/paper/2301.10297/) (2023)
- [Linking In-context Learning in Transformers to Human Episodic Memory](https://mlsanity.com/paper/2405.14992/) (2024)
- [Spectral Ranking using Seriation](https://mlsanity.com/paper/1406.5370/) (2014)
- [Position Information Emerges in Causal Transformers Without Positional Encodings via Similarity of Nearby Embeddings](https://mlsanity.com/paper/2501.00073/) (2024)
- [Large Language Models as Model Organisms for Human Associative Learning](https://mlsanity.com/paper/2510.21408/) (2025)

## Abstract (arXiv)

Human episodic memory supports the retrieval of experiences that unfold over extended timescales, yet the computational mechanisms underlying this ability remain debated due to the limited mechanistic accessibility in long-term memory experiments in humans. Long-context LLMs may offer promising ways to reveal plausible computational mechanisms that drive this type of retrieval. Here, we investigate whether and how LLMs capture the core behavioral signatures of episodic memory via a temporal order memory task. Using a new dataset of human behavior based on memory of a full-length novel, we show that models exhibit the same characteristic distance effect observed in humans on this task. We next apply long-context mechanistic interpretability analyses to uncover how models solve this task, and find that model performance relies on a one-dimensional temporal code that is reinstated during retrieval by a single time-reinstatement attention head. These findings support temporal context reinstatement as an important mechanism for episodic-like temporal-order memory in LLMs, offering new insights into how temporal aspects of long-term episodic memory may be instantiated in both artificial and biological systems.