mlsanityData to 7 Oct 2026

Allocating Recurrent Compute in Looped Language Models

arXiv 2608.18230 ↗First posted 18 Aug 2026cs.LGcited by 4

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Looped language models improve reasoning and knowledge manipulation by applying shared computation repeatedly. Existing systems usually repeat an entire layer stack, although a mixer and a dense feed-forward network (FFN) perform different operations and have different costs. We ask a narrower question: what should loop? We view recurrence as repeated composition of a state update and argue that an application is valuable when it exposes a new cross-position influence direction that remains observable at the task readout. Iterative Transport Rank (ITR) describes the cumulative influence trajectory; marginal ITR describes the nonredundant influence contributed by successive applications. This view motivates MixerLoop, which repeats each Gated DeltaNet mixer while applying its dense FFN once. We compare MixerLoop with no recurrence and full-block recurrence at 15M and 110M parameters under the same data, initialization, and architecture. A finite context-off intervention tests whether later mixer applications produce distinct, non-negligible, and beneficial changes at the final language-model readout. MixerLoop surpasses FullLoop on aggregate CORE at 15M and retains 41.5% of its CORE improvement at 110M while reducing recurrent-backbone projection FLOPs by 45.9%. These results show that the benefits of recurrent depth can be retained without repeatedly executing the dense FFN.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 22 of its 25 references that are in the map. First posted on arXiv 18 Aug 2026.

Builds on

22 of 25 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language Models26 May
  2. 2026LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulation11 Feb
  3. 2025Scaling Latent Reasoning via Looped Language Models29 Oct
  4. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformers24 Feb
  5. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach7 Feb

Other references · 17

  1. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  2. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  3. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 594
  4. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 483
  5. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  6. 2023Looped Transformers are Better at Learning Learning Algorithmscited by 135
  7. 2024Looped Transformers for Length Generalizationcited by 91
  8. 2025Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-trainingcited by 62
  9. 2024MoEUT: Mixture-of-Experts Universal Transformerscited by 55
  10. 2026Gated DeltaNet-2: Decoupling Erase and Write in Linear Attentioncited by 24
  11. 2025Skip a Layer or Loop it? Test-Time Depth Adaptation of Pretrained LLMscited by 14
  12. 2026Depth-Recurrent Attention Mixtures: Giving Latent Reasoning the Attention it Deservescited by 11
5 more
  1. 2026Memory-Efficient Looped Transformer: Decoupling Compute from Memory in Looped Language Modelscited by 11
  2. 2026Sparse Layers are Critical to Scaling Looped Language Modelscited by 10
  3. 2026LoopMoE: Unifying Iterative Computation with Mixture-of-Experts for Language Modelingcited by 7
  4. 2026LT2: Linear-Time Looped Transformerscited by 5
  5. 2026Sparse Growing Transformer: Training-Time Sparse Depth Allocation via Progressive Attention Loopingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Beyond Depth Truncation: Controlled Evaluation of Depth Utilization in Recursive Language Modelscited by 0
  2. 2026Improving Test-Time Scaling with Adaptive Looped Transformerscited by 0
  3. 2026What Makes Recurrence Effective in Looped Language Models?cited by 0
  4. 2026LatentHarness: Learning Latent Actions for Memory and Reasoning via Counterfactual Policy Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation for Looped Language Models
  2. 2026A Mechanistic Analysis of Looped Reasoning Language Models
  3. 2026RecurTrace: Adaptive Latent Reasoning with Loop-Time Memory
  4. 2026Trading Complexity for Expressivity Through Structured Generalized Linear Token Mixing
  5. 2026Scheduling Recursive Reasoning in Looped Transformers
  6. 2026Towards Looped Models Done Right, Part II: Rethinking at Fixed Points
  7. 2026Loop Dropout: Regularizing Shared Updates in Looped Language Models
  8. 2026Closing the Loop: Practical Training Recipes for Looped Language Models