# Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers

arXiv: https://arxiv.org/abs/2606.18206 · HTML: https://mlsanity.com/paper/2606.18206/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 67 references that are in the map.
- First posted on arXiv 16 Jun 2026.

## Builds on (most cited first, 30 of 53)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [Universal Transformers](https://mlsanity.com/paper/1807.03819/) (2018)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Adaptive Computation Time for Recurrent Neural Networks](https://mlsanity.com/paper/1603.08983/) (2016)
- [Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth](https://mlsanity.com/paper/2103.03404/) (2021)
- [Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach](https://mlsanity.com/paper/2502.05171/) (2025)
- [Resurrecting Recurrent Neural Networks for Long Sequences](https://mlsanity.com/paper/2303.06349/) (2023)
- [Reasoning with Latent Thoughts: On the Power of Looped Transformers](https://mlsanity.com/paper/2502.17416/) (2025)
- [Looped Transformers as Programmable Computers](https://mlsanity.com/paper/2301.13196/) (2023)
- [Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC](https://mlsanity.com/paper/2302.11552/) (2023)
- [The Illusion of State in State-Space Models](https://mlsanity.com/paper/2404.08819/) (2024)
- [Hierarchical Reasoning Model](https://mlsanity.com/paper/2506.21734/) (2025)
- [The Lipschitz Constant of Self-Attention](https://mlsanity.com/paper/2006.04710/) (2020)
- [PonderNet: Learning to Ponder](https://mlsanity.com/paper/2107.05407/) (2021)
- [Scaling Latent Reasoning via Looped Language Models](https://mlsanity.com/paper/2510.25741/) (2025)
- [Looped Transformers are Better at Learning Learning Algorithms](https://mlsanity.com/paper/2311.12424/) (2023)
- [Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse](https://mlsanity.com/paper/2206.03126/) (2022)
- [Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation](https://mlsanity.com/paper/2507.10524/) (2025)
- [Neural GPUs Learn Algorithms](https://mlsanity.com/paper/1511.08228/) (2015)
- [Looped Transformers for Length Generalization](https://mlsanity.com/paper/2409.15647/) (2024)
- [JFB: Jacobian-Free Backpropagation for Implicit Networks](https://mlsanity.com/paper/2103.12803/) (2021)
- [Scaling Exponents Across Parameterizations and Optimizers](https://mlsanity.com/paper/2407.05872/) (2024)
- [On Training Implicit Models](https://mlsanity.com/paper/2111.05177/) (2021)

## Built on by (most cited first, 28 of 28)

- [Self-conditioned Flow Map Language Models via Fixed-point Flows](https://mlsanity.com/paper/2607.00714/) (2026)
- [Thinking with Looped Flows](https://mlsanity.com/paper/2609.11801/) (2026)
- [Loop the Loopies!](https://mlsanity.com/paper/2607.16051/) (2026)
- [Flow Reasoning Models: Turning Flows Into Efficient Recurrent Reasoners](https://mlsanity.com/paper/2606.29150/) (2026)
- [Full-bandwidth transformer](https://mlsanity.com/paper/2608.08888/) (2026)
- [Energy-guided Recursive Model](https://mlsanity.com/paper/2607.10128/) (2026)
- [Fractal basins trap latent reasoning](https://mlsanity.com/paper/2609.04963/) (2026)
- [Improving Test-Time Scaling with Adaptive Looped Transformers](https://mlsanity.com/paper/2609.35748/) (2026)
- [Closing the Loop: Practical Training Recipes for Looped Language Models](https://mlsanity.com/paper/2610.00673/) (2026)
- [Recursive Vision Language Models for General Symbolic Reasoning](https://mlsanity.com/paper/2608.01534/) (2026)
- [If It's Not Buggy, Don't Fix It: On the Dynamics of Iterative Bug-fixing with LLMs](https://mlsanity.com/paper/2609.10123/) (2026)
- [Storing Is Not Remembering: LSTM-UT and Bounded Gated Memory for Looped Transformers](https://mlsanity.com/paper/2609.19521/) (2026)
- [LoopICL: Looping a single transformer block to solve tabular tasks](https://mlsanity.com/paper/2609.36108/) (2026)
- [What Makes Recurrence Effective in Looped Language Models?](https://mlsanity.com/paper/2609.36636/) (2026)
- [Scheduling Recursive Reasoning in Looped Transformers](https://mlsanity.com/paper/2609.36653/) (2026)
- [Looped Actor: Depth-Recurrent Reasoning Models for Reinforcement Learning](https://mlsanity.com/paper/2609.37432/) (2026)
- [Beyond Simulation: Retain-and-Repair Neural Operators for Real-World Adaptation](https://mlsanity.com/paper/2609.39387/) (2026)
- [What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling](https://mlsanity.com/paper/2609.39967/) (2026)
- [On-the-fly Weight Generation: A Hypernetwork Proof of Concept on ARC-1D](https://mlsanity.com/paper/2610.00820/) (2026)
- [Looping Beyond Twice: A Scalable Recipe for Looped Mixture-of-Experts](https://mlsanity.com/paper/2610.01153/) (2026)
- [ALoDLM: Adaptively Looped Diffusion Language Models](https://mlsanity.com/paper/2610.04198/) (2026)
- [Towards Looped Models Done Right, Part II: Rethinking at Fixed Points](https://mlsanity.com/paper/2610.06833/) (2026)
- [Unrolled Flow Models for Reasoning](https://mlsanity.com/paper/2610.09759/) (2026)
- [Scaling to Tens of Thousands of Test-Time Iterations with Loop-Native Attention Residuals](https://mlsanity.com/paper/2610.11570/) (2026)
- [Pretraining and Test-Time Training Budgets Matter for ARC: A Controlled Study of VARC and TRM](https://mlsanity.com/paper/2610.13660/) (2026)
- [Rotate the Loop: Blockwise Conformality for Looped Transformers](https://mlsanity.com/paper/2610.14030/) (2026)
- [Train for Many, Update with One: Improving Multi-Sample Reasoning via Fenchel-Bregman Posterior Learning](https://mlsanity.com/paper/2610.17735/) (2026)
- [Looped Reasoning Finishes Earlier Than You Think](https://mlsanity.com/paper/2610.18174/) (2026)

## Abstract (arXiv)

Looped architectures provide an inductive bias toward learning step-by-step procedures for tasks that require compositional reasoning. The number of effective layers reached by looping determines the quality of the solution these models find. Like deep architectures, looped architectures are prone to a signal propagation problem induced by depth as the halting decision is postponed. In this paper, we address this signal propagation issue using pre-norm layers and residual scaling. Building on these architectural modifications, we propose FPRM, a Transformer-based Fixed-Point Reasoning Model that uses fixed-point convergence as an end-to-end halting mechanism in a looped architecture. We show that fixed-point halting allows FPRM to adapt its compute to task difficulty. FPRM is effective on common reasoning benchmarks, namely Sudoku, Maze, state-tracking, and ARC-AGI.