mlsanityData to 7 Oct 2026

When Recursive Models Finish Computing

arXiv 2609.26487 ↗First posted 22 Sep 2026cs.LG cs.AI cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

Recursive models can continue updating their latent states beyond their nominal inference budget, so an incorrect output at that budget does not show whether computation is unfinished or has entered a persistently unsuccessful regime. We study the dynamics of completion in attention- and MLP-based Tiny Recursive Models (TRMs) on 1,000 hard Sudoku puzzles. Extending recurrence from the nominal 16 steps to 512 steps increases cumulative exact-solve accuracy from 59.2% to 87.5% for the attention model and from 74.4% to 91.9% for the MLP model, solving more than two-thirds of the puzzles unsolved in the nominal budget. Across both architectures, latent-state motion drops sharply after the first exact solution. Completed states are typically locally contractive along the trajectory direction, even though the same local Jacobian retains strongly expanding directions. We characterize this phenomenon as trajectory-conditioned anisotropic stability. Perturbation experiments confirm this directional stability across both models. The multi-step fate of the maximally expanding direction differs: it is absorbed within 16 steps in the attention model but persists longer in the MLP model. The anisotropic-stability pattern also holds for a second attention checkpoint. Together, these results distinguish nominal-budget failure from completed computation and identify a common dynamical signature of completion across two recurrent architectures.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 35 references that are in the map. First posted on arXiv 22 Sep 2026.

Builds on

23 of 35 references.

References · 23

  1. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  2. 2015Automatic differentiation in machine learning: a surveycited by 846
  3. 2018Universal Transformerscited by 613
  4. 2019Deep Equilibrium Modelscited by 603
  5. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  6. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
  7. 2023Looped Transformers as Programmable Computerscited by 224
  8. 2019SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solvercited by 173
  9. 2025Hierarchical Reasoning Modelcited by 167
  10. 2021Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networkscited by 108
  11. 2017Recurrent Relational Networkscited by 71
  12. 2025Grokking at the Edge of Numerical Stabilitycited by 41
11 more
  1. 2019Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamicscited by 38
  2. 2020Lyapunov spectra of chaotic recurrent neural networkscited by 37
  3. 2022Path Independent Equilibrium Models Can Better Exploit Test-Time Computationcited by 33
  4. 2020On Lyapunov Exponents for RNNs: Understanding Information Propagation Using Dynamical Systems Toolscited by 31
  5. 2019Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamicscited by 28
  6. 2025Sudoku-Bench: Evaluating creative reasoning with Sudoku variantscited by 21
  7. 2026Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoningcited by 12
  8. 2026Are Your Reasoning Models Reasoning or Guessing? A Mechanistic Analysis of Hierarchical Reasoning Modelscited by 9
  9. 2024On Logical Extrapolation for Mazes with Recurrent and Implicit Networkscited by 7
  10. 2026Think Shallow, Solve Deep: Controlling Recurrent Dynamics for Reliable Test-Time Depthcited by 5
  11. 2026Fractal basins trap latent reasoningcited by 2

Built on by

Not cited yet.