mlsanityData to 7 Oct 2026

Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers

arXiv 2606.18206 ↗First posted 16 Jun 2026cs.AIcited by 21

Not on the map. Why?

Abstract (arXiv)

Looped architectures provide an inductive bias toward learning step-by-step procedures for tasks that require compositional reasoning. The number of effective layers reached by looping determines the quality of the solution these models find. Like deep architectures, looped architectures are prone to a signal propagation problem induced by depth as the halting decision is postponed. In this paper, we address this signal propagation issue using pre-norm layers and residual scaling. Building on these architectural modifications, we propose FPRM, a Transformer-based Fixed-Point Reasoning Model that uses fixed-point convergence as an end-to-end halting mechanism in a looped architecture. We show that fixed-point halting allows FPRM to adapt its compute to task difficulty. FPRM is effective on common reasoning benchmarks, namely Sudoku, Maze, state-tracking, and ARC-AGI.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 53 of its 67 references that are in the map. First posted on arXiv 16 Jun 2026.

Builds on

53 of 67 references.

References · 53

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  6. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  7. 2020On Layer Normalization in the Transformer Architecturecited by 906
  8. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  9. 2018Universal Transformerscited by 613
  10. 2019Deep Equilibrium Modelscited by 603
  11. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  12. 2021Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depthcited by 411
38 more
  1. 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 397
  2. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
  3. 2023Looped Transformers as Programmable Computerscited by 224
  4. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 223
  5. 2023Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMCcited by 216
  6. 2024The Illusion of State in State-Space Modelscited by 181
  7. 2020The Lipschitz Constant of Self-Attentioncited by 171
  8. 2025Hierarchical Reasoning Modelcited by 167
  9. 2021PonderNet: Learning to Pondercited by 156
  10. 2025Scaling Latent Reasoning via Looped Language Modelscited by 145
  11. 2022Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapsecited by 136
  12. 2023Looped Transformers are Better at Learning Learning Algorithmscited by 135
  13. 2025Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computationcited by 120
  14. 2015Neural GPUs Learn Algorithmscited by 118
  15. 2024Looped Transformers for Length Generalizationcited by 91
  16. 2021JFB: Jacobian-Free Backpropagation for Implicit Networkscited by 84
  17. 2021On Training Implicit Modelscited by 73
  18. 2021Stabilizing Equilibrium Models by Jacobian Regularizationcited by 71
  19. 2024Scaling Exponents Across Parameterizations and Optimizerscited by 71
  20. 2024RecurrentGemma: Moving Past Transformers for Efficient Open Language Modelscited by 61
  21. 2015Structured Prediction Energy Networkscited by 59
  22. 2025The Curse of Depth in Large Language Modelscited by 58
  23. 2026LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulationcited by 48
  24. 2024Learning Iterative Reasoning through Energy Diffusioncited by 36
  25. 2026Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformerscited by 34
  26. 2017End-to-End Learning for Structured Prediction Energy Networkscited by 33
  27. 2022Path Independent Equilibrium Models Can Better Exploit Test-Time Computationcited by 33
  28. 2022Learning Iterative Reasoning through Energy Minimizationcited by 32
  29. 2025Peri-LN: Revisiting Normalization Layer in the Transformer Architecturecited by 32
  30. 2025Energy-Based Transformers are Scalable Learners and Thinkerscited by 30
  31. 2026A Mechanistic Analysis of Looped Reasoning Language Modelscited by 27
  32. 2025ARC Is a Vision Problem!cited by 23
  33. 2026LoopViT: Scaling Visual ARC with Looped Transformerscited by 14
  34. 2025Hierarchical Reasoning Models: Perspectives and Misconceptionscited by 13
  35. 2026Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoningcited by 12
  36. 2025Universal Reasoning Modelcited by 11
  37. 2026Solve the Loop: Attractor Models for Language and Reasoningcited by 11
  38. 2025Performative Thinking? The Brittle Correlation Between CoT Length and Problem Complexitycited by 9

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 21.

21 papers.

Sort:

By area:

  1. 2026Self-conditioned Flow Map Language Models via Fixed-point Flowscited by 9
  2. 2026Thinking with Looped Flowscited by 6
  3. 2026Flow Reasoning Models: Turning Flows Into Efficient Recurrent Reasonerscited by 5
  4. 2026Loop the Loopies!cited by 4
  5. 2026Full-bandwidth transformercited by 3
  6. 2026Fractal basins trap latent reasoningcited by 2
  7. 2026Energy-guided Recursive Modelcited by 1
  8. 2026Recursive Vision Language Models for General Symbolic Reasoningcited by 0
  9. 2026If It's Not Buggy, Don't Fix It: On the Dynamics of Iterative Bug-fixing with LLMscited by 0
  10. 2026Storing Is Not Remembering: LSTM-UT and Bounded Gated Memory for Looped Transformerscited by 0
  11. 2026Improving Test-Time Scaling with Adaptive Looped Transformerscited by 0
  12. 2026LoopICL: Looping a single transformer block to solve tabular taskscited by 0
9 more
  1. 2026What Makes Recurrence Effective in Looped Language Models?cited by 0
  2. 2026Scheduling Recursive Reasoning in Looped Transformerscited by 0
  3. 2026Looped Actor: Depth-Recurrent Reasoning Models for Reinforcement Learningcited by 0
  4. 2026Beyond Simulation: Retain-and-Repair Neural Operators for Real-World Adaptationcited by 0
  5. 2026What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scalingcited by 0
  6. 2026On-the-fly Weight Generation: A Hypernetwork Proof of Concept on ARC-1Dcited by 0
  7. 2026Looping Beyond Twice: A Scalable Recipe for Looped Mixture-of-Expertscited by 0
  8. 2026Unrolled Flow Models for Reasoningcited by 0
  9. 2026Scaling to Tens of Thousands of Test-Time Iterations with Loop-Native Attention Residualscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Adaptive Loops and Memory in Transformers: Think Harder or Know More?
  2. 2026Memory-Efficient Looped Transformer: Decoupling Compute from Memory in Looped Language Models
  3. 2026Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
  4. 2026Reasoning on the Simplex: Geometric Fixed-Point Models
  5. 2026LoopMTP: A looped transformer guided by latent multi-token prediction
  6. 2026Training-Free Looped Transformers
  7. 2026T-LoopFormer: Token-Level Elastic-Depth Looped Transformers for Latent Reasoning with Dynamic Routing
  8. 2026LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Models