mlsanityData to 14 Oct 2026

Residual Streams Read, Recurrent States Remember: The Global Workspace in Mamba Models

arXiv 2609.32102 ↗First posted 26 Sep 2026cs.AIcited by 0

On the map: State-space sequence models (S4, Mamba) → Core SSM architectures for language

Abstract (arXiv)

Can the global-workspace account of transformer representations extend to state-space language models? We fit Jacobian lenses to the residual streams and recurrent states of Mamba-1, Mamba-2 and Mamba-3, using the original 1000-prompt recipe. Joint residual--state readouts improve recovery of known intermediate concepts over the residual lens on at least five of six task families in every tested Mamba checkpoint. On Mamba-2, state alone exceeds residual and logit lenses on all six families; a normalised joint readout improves on both components on five. Temporal maps and word-list experiments show earlier content remaining state-readable as residual visibility changes. We also propose sign-guarded steering, which improves target top-five success over coordinate exchange on matched verbal-report trials in five models. Recurrent state alone supports this verbal access. These gains do not extend consistently to relational answers: guarded edits often output the edited concept itself, and Mamba-3's joint edits can disrupt successful state-only redirection. Recurrent state thus provides a complementary carrier of workspace content, whose recovery, persistence and causal uses require separate measurements.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 8 of its 10 references that are in the map. First posted on arXiv 26 Sep 2026.

Builds on

8 of 10 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026Mamba-3: Improved Sequence Modeling using State Space Principles16 Mar
  2. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality31 May
  3. 2024Does Transformer Interpretability Transfer to RNNs?9 Apr
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spaces1 Dec

Other references · 4

  1. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  2. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  3. 2026Verbalizable Representations Form a Global Workspace in Language Modelscited by 75
  4. 2026Looped Transformers under the Jacobian Lens: Does the Global Workspace Survive Recurrence?cited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025LongMamba: Enhancing Mamba's Long Context Capabilities via Training-Free Receptive Field Enlargement
  2. 2024Understanding and Mitigating Bottlenecks of State Space Models through the Lens of Recency and Over-smoothing
  3. 2026Comparing Latent Concept Formation in State Space Models and Transformers via Sparse Autoencoders
  4. 2024Locating and Editing Factual Associations in Mamba
  5. 2026Interpreting and Steering State-Space Models via Activation Subspace Bottlenecks
  6. 2024An Empirical Study of Mamba-based Language Models
  7. 2025Understanding the Skill Gap in Recurrent Language Models: The Role of the Gather-and-Aggregate Mechanism
  8. 2026The Ignition Index: Measuring Global Workspace Dynamics in Language Models