# Allocating Recurrent Compute in Looped Language Models

arXiv: https://arxiv.org/abs/2608.18230 · HTML: https://mlsanity.com/paper/2608.18230/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 22 of its 25 references that are in the map.
- First posted on arXiv 18 Aug 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 22 of 22)

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Retentive Network: A Successor to Transformer for Large Language Models](https://mlsanity.com/paper/2307.08621/) (2023)
- [Gated Delta Networks: Improving Mamba2 with Delta Rule](https://mlsanity.com/paper/2412.06464/) (2024)
- [Linear Transformers Are Secretly Fast Weight Programmers](https://mlsanity.com/paper/2102.11174/) (2021)
- [Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach](https://mlsanity.com/paper/2502.05171/) (2025)
- [Reasoning with Latent Thoughts: On the Power of Looped Transformers](https://mlsanity.com/paper/2502.17416/) (2025)
- [Scaling Latent Reasoning via Looped Language Models](https://mlsanity.com/paper/2510.25741/) (2025)
- [Looped Transformers are Better at Learning Learning Algorithms](https://mlsanity.com/paper/2311.12424/) (2023)
- [Looped Transformers for Length Generalization](https://mlsanity.com/paper/2409.15647/) (2024)
- [Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training](https://mlsanity.com/paper/2504.13161/) (2025)
- [LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulation](https://mlsanity.com/paper/2602.11451/) (2026)
- [MoEUT: Mixture-of-Experts Universal Transformers](https://mlsanity.com/paper/2405.16039/) (2024)
- [Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention](https://mlsanity.com/paper/2605.22791/) (2026)
- [Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language Models](https://mlsanity.com/paper/2605.26733/) (2026)
- [Skip a Layer or Loop it? Test-Time Depth Adaptation of Pretrained LLMs](https://mlsanity.com/paper/2507.07996/) (2025)
- [Memory-Efficient Looped Transformer: Decoupling Compute from Memory in Looped Language Models](https://mlsanity.com/paper/2605.07721/) (2026)
- [Depth-Recurrent Attention Mixtures: Giving Latent Reasoning the Attention it Deserves](https://mlsanity.com/paper/2601.21582/) (2026)
- [Sparse Layers are Critical to Scaling Looped Language Models](https://mlsanity.com/paper/2605.09165/) (2026)
- [LoopMoE: Unifying Iterative Computation with Mixture-of-Experts for Language Modeling](https://mlsanity.com/paper/2606.04438/) (2026)
- [LT2: Linear-Time Looped Transformers](https://mlsanity.com/paper/2605.20670/) (2026)
- [Sparse Growing Transformer: Training-Time Sparse Depth Allocation via Progressive Attention Looping](https://mlsanity.com/paper/2603.23998/) (2026)

## Built on by (most cited first, 4 of 4)

- [Beyond Depth Truncation: Controlled Evaluation of Depth Utilization in Recursive Language Models](https://mlsanity.com/paper/2609.19934/) (2026)
- [Improving Test-Time Scaling with Adaptive Looped Transformers](https://mlsanity.com/paper/2609.35748/) (2026)
- [What Makes Recurrence Effective in Looped Language Models?](https://mlsanity.com/paper/2609.36636/) (2026)
- [LatentHarness: Learning Latent Actions for Memory and Reasoning via Counterfactual Policy Distillation](https://mlsanity.com/paper/2609.39740/) (2026)

## Abstract (arXiv)

Looped language models improve reasoning and knowledge manipulation by applying shared computation repeatedly. Existing systems usually repeat an entire layer stack, although a mixer and a dense feed-forward network (FFN) perform different operations and have different costs. We ask a narrower question: what should loop? We view recurrence as repeated composition of a state update and argue that an application is valuable when it exposes a new cross-position influence direction that remains observable at the task readout. Iterative Transport Rank (ITR) describes the cumulative influence trajectory; marginal ITR describes the nonredundant influence contributed by successive applications. This view motivates MixerLoop, which repeats each Gated DeltaNet mixer while applying its dense FFN once. We compare MixerLoop with no recurrence and full-block recurrence at 15M and 110M parameters under the same data, initialization, and architecture. A finite context-off intervention tests whether later mixer applications produce distinct, non-negligible, and beneficial changes at the final language-model readout. MixerLoop surpasses FullLoop on aggregate CORE at 15M and retains 41.5% of its CORE improvement at 110M while reducing recurrent-backbone projection FLOPs by 45.9%. These results show that the benefits of recurrent depth can be retained without repeatedly executing the dense FFN.