mlsanityData to 14 Oct 2026

Looped Transformers under the Jacobian Lens: Does the Global Workspace Survive Recurrence?

arXiv 2609.01924 ↗First posted 1 Sep 2026cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Recent work identifies a mid-depth band of verbalisable, causally potent representations in a standard feedforward transformer --- a functional analogue of a global workspace. Whether the same workspace functionality emerges when depth is implemented through recurrence rather than a stack of distinct layers remains unknown. Looped and depth-recurrent transformers provide a direct test of this question because they reuse the same weights across depth. We extend the Jacobian lens to iterated architectures using a virtual-unrolling adapter. We apply the full workspace suite --- lens fitting, readout, and eleven causal experiment families --- to Ouro-2.6B (48 layers looped 4 times, deeply supervised) and Huginn-0125 (a 4-layer core recurred 16 times, trained for latent reasoning), using Qwen3.6-27B (64 untied layers) as the standard baseline. We find that a workspace forms in the iterated part of each architecture, but that recurrence changes how it can be accessed. Ouro reconstructs workspace content in every loop, and linear transport cannot carry that content across loop boundaries; writes and ablations must therefore span every remaining loop. Huginn carries content forward across all sixteen recurrences, while reads, writes, and ablations act only within a sliding window of roughly two recurrences. Whether newly injected content can be verbalised tracks explicit per-iteration supervision; whether existing content can be steered does not.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 11 of its 20 references that are in the map. First posted on arXiv 1 Sep 2026.

Builds on

11 of 20 references.

References · 11

  1. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  2. 2018Universal Transformerscited by 647
  3. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  4. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 415
  5. 2023Looped Transformers as Programmable Computerscited by 237
  6. 2025Scaling Latent Reasoning via Looped Language Modelscited by 165
  7. 2023Looped Transformers are Better at Learning Learning Algorithmscited by 149
  8. 2023Consciousness in Artificial Intelligence: Insights from the Science of Consciousnesscited by 104
  9. 2026Verbalizable Representations Form a Global Workspace in Language Modelscited by 75
  10. 2026A Mechanistic Analysis of Looped Reasoning Language Modelscited by 33
  11. 2025Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformercited by 27

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.