Timesteps of Mamba Align with Human Reading Times
On the map: State-space sequence models (S4, Mamba) → Core SSM architectures for language
Abstract (arXiv)
This study demonstrates an alignment of per-word processing time in a popular state-space language model Mamba and human readers. In Mamba, the recurrent state transition at each layer conceptually takes some duration of time, the discretization timestep Deltat, determined dynamically in response to the input. Using a naturalistic reading dataset, we show that the per-word timestep from Mamba is a significant predictor of human reading times, and remains significant even when known predictors such as GPT-2 surprisal are controlled for. We further suggest, through formal analysis of Mamba's architecture and internal dynamics, that Mamba can serve as a new, valuable lens to look at human real-time language processing with ever-updated memory, because it allows us to look at how each module (layer) weighs short- and long-term information retention, and how noise may interact with dynamic, continuous memory representation. Code is available online.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 12 of its 27 references that are in the map. First posted on arXiv 29 Jun 2026.
Builds on
12 of 27 references.
Closest ancestors · 4
Same direction, newest first
- 2025Mamba Knockout for Unraveling Factual Information Flow30 May
- 2024Falcon Mamba: The First Competitive Attention-free 7B Language Model7 Oct
- 2024Locating and Editing Factual Associations in Mamba4 Apr
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spaces1 Dec
Other references · 8
- 2022Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?cited by 113
- 2023Testing the Predictions of Surprisal Theory in 11 Languagescited by 77
- 2021Lower Perplexity is Not Always Human-Likecited by 52
- 2017The Natural Stories Corpuscited by 40
- 2022Context Limitations Make Neural Language Models More Human-Likecited by 34
- 2025Large Language Models Are Human-Like Internallycited by 29
- 2021Accounting for Agreement Phenomena in Sentence Comprehension with Transformer Language Models: Effects of Similarity-based Interference on Surprisal and Attentioncited by 27
- 2025If Attention Serves as a Cognitive Model of Human Memory Retrieval, What is the Plausible Memory Representation?cited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025MemMamba: Rethinking Memory Patterns in State Space Model
- 2025Emergence of Primacy and Recency Effect in Mamba: A Mechanistic Point of View
- 2024Revealing and Mitigating the Local Pattern Shortcuts of Mamba
- 2026Understanding Mamba-3 Through Oscillatory Dynamics
- 2026STRUCTURALCOST: A controlled reading time dataset for modeling human sentence processing difficulty
- 2025LongMamba: Enhancing Mamba's Long Context Capabilities via Training-Free Receptive Field Enlargement
- 2026Mamba with Hierarchical Memory: Solving Representation Bottleneck in Long Sequence Modeling
- 2024ReMamba: Equip Mamba with Effective Long-Sequence Modeling