# Loop as a Bridge: Can Looped Transformers Truly Link Representation Space and Natural Language Outputs?

arXiv: https://arxiv.org/abs/2601.10242 · HTML: https://mlsanity.com/paper/2601.10242/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 29 references that are in the map.
- First posted on arXiv 15 Jan 2026.

## Builds on (most cited first, 24 of 24)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset](https://mlsanity.com/paper/2307.04657/) (2023)
- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning](https://mlsanity.com/paper/2504.11456/) (2025)
- [Scaling Latent Reasoning via Looped Language Models](https://mlsanity.com/paper/2510.25741/) (2025)
- [Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation](https://mlsanity.com/paper/2507.10524/) (2025)
- [Are self-explanations from Large Language Models faithful?](https://mlsanity.com/paper/2401.07927/) (2024)
- [Emergent Introspective Awareness in Large Language Models](https://mlsanity.com/paper/2601.01828/) (2026)
- [Latent Guard: a Safety Framework for Text-to-image Generation](https://mlsanity.com/paper/2404.08031/) (2024)
- [Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence](https://mlsanity.com/paper/2511.07384/) (2025)
- [PonderLM: Pretraining Language Models to Ponder in Continuous Space](https://mlsanity.com/paper/2505.20674/) (2025)
- [PonderLM-2: Pretraining LLM with Latent Thoughts in Continuous Space](https://mlsanity.com/paper/2509.23184/) (2025)
- [Efficient Detection of Toxic Prompts in Large Language Models](https://mlsanity.com/paper/2408.11727/) (2024)
- [Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoring](https://mlsanity.com/paper/2502.05242/) (2025)

## Built on by (most cited first, 8 of 8)

- [LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulation](https://mlsanity.com/paper/2602.11451/) (2026)
- [LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Models](https://mlsanity.com/paper/2605.11011/) (2026)
- [Training-Free Looped Transformers](https://mlsanity.com/paper/2605.23872/) (2026)
- [LT2: Linear-Time Looped Transformers](https://mlsanity.com/paper/2605.20670/) (2026)
- [LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling](https://mlsanity.com/paper/2606.18023/) (2026)
- [T-LoopFormer: Token-Level Elastic-Depth Looped Transformers for Latent Reasoning with Dynamic Routing](https://mlsanity.com/paper/2609.15160/) (2026)
- [Introspection Fine-Tuning (IFT): Training Small LLMs to Introspect](https://mlsanity.com/paper/2607.14111/) (2026)
- [Improving Test-Time Scaling with Adaptive Looped Transformers](https://mlsanity.com/paper/2609.35748/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) often exhibit a gap between their internal knowledge and their explicit linguistic outputs. In this report, we empirically investigate whether Looped Transformers (LTs)--architectures that increase computational depth by iterating shared layers--can bridge this gap by utilizing their iterative nature as a form of introspection. Our experiments reveal that while increasing loop iterations narrows the gap, it is partly driven by a degradation of their internal knowledge carried by representations. Moreover, another empirical analysis suggests that current LTs' ability to perceive representations does not improve across loops; it is only present in the final loop. These results suggest that while LTs offer a promising direction for scaling computational depth, they have yet to achieve the introspection required to truly link representation space and natural language.