Looped Transformers under the Jacobian Lens: Does the Global Workspace Survive Recurrence?
Not on the map. Why?
Abstract (arXiv)
Recent work identifies a mid-depth band of verbalisable, causally potent representations in a standard feedforward transformer --- a functional analogue of a global workspace. Whether the same workspace functionality emerges when depth is implemented through recurrence rather than a stack of distinct layers remains unknown. Looped and depth-recurrent transformers provide a direct test of this question because they reuse the same weights across depth. We extend the Jacobian lens to iterated architectures using a virtual-unrolling adapter. We apply the full workspace suite --- lens fitting, readout, and eleven causal experiment families --- to Ouro-2.6B (48 layers looped 4 times, deeply supervised) and Huginn-0125 (a 4-layer core recurred 16 times, trained for latent reasoning), using Qwen3.6-27B (64 untied layers) as the standard baseline. We find that a workspace forms in the iterated part of each architecture, but that recurrence changes how it can be accessed. Ouro reconstructs workspace content in every loop, and linear transport cannot carry that content across loop boundaries; writes and ablations must therefore span every remaining loop. Huginn carries content forward across all sixteen recurrences, while reads, writes, and ablations act only within a sliding window of roughly two recurrences. Whether newly injected content can be verbalised tracks explicit per-iteration supervision; whether existing content can be steered does not.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 11 of its 20 references that are in the map. First posted on arXiv 1 Sep 2026.
Builds on
11 of 20 references.
References · 11
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
- 2018Universal Transformerscited by 647
- 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
- 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 415
- 2023Looped Transformers as Programmable Computerscited by 237
- 2025Scaling Latent Reasoning via Looped Language Modelscited by 165
- 2023Looped Transformers are Better at Learning Learning Algorithmscited by 149
- 2023Consciousness in Artificial Intelligence: Insights from the Science of Consciousnesscited by 104
- 2026Verbalizable Representations Form a Global Workspace in Language Modelscited by 75
- 2026A Mechanistic Analysis of Looped Reasoning Language Modelscited by 33
- 2025Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformercited by 27
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Improving Test-Time Scaling with Adaptive Looped Transformerscited by 1
- 2026Residual Streams Read, Recurrent States Remember: The Global Workspace in Mamba Modelscited by 0
- 2026Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes Itcited by 0
- 2026Rotate the Loop: Blockwise Conformality for Looped Transformerscited by 0
- 2026Monitoring latent reasoning in looped language modelscited by 0
- 2026Learning to Compute Is Not Learning to Retain or Reuse in Looped Transformerscited by 0
- 2026Learning to Compute Is Not Learning to Retain or Reuse in Looped Transformers14 Oct 2026
- 2026Monitoring latent reasoning in looped language models13 Oct 2026
- 2026Rotate the Loop: Blockwise Conformality for Looped Transformers30 Sep 2026
- 2026Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It29 Sep 2026
- 2026Improving Test-Time Scaling with Adaptive Looped Transformers28 Sep 2026
- 2026Residual Streams Read, Recurrent States Remember: The Global Workspace in Mamba Models26 Sep 2026