mlsanityData to 14 Oct 2026

Scaling Latent Reasoning via Looped Language Models

arXiv 2510.25741 ↗First posted 29 Oct 2025cs.CLcited by 165

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Modern LLMs are trained to "think" primarily via explicit text generation, such as chain-of-thought (CoT), which defers reasoning to post-training and under-leverages pre-training data. We present and open-source Ouro, named after the recursive Ouroboros, a family of pre-trained Looped Language Models (LoopLM) that instead build reasoning into the pre-training phase through (i) iterative computation in latent space, (ii) an entropy-regularized objective for learned depth allocation, and (iii) scaling to 7.7T tokens. Ouro 1.4B and 2.6B models enjoy superior performance that match the results of up to 12B SOTA LLMs across a wide range of benchmarks. Through controlled experiments, we show this advantage stems not from increased knowledge capacity, but from superior knowledge manipulation capabilities. We also show that LoopLM yields reasoning traces more aligned with final outputs than explicit CoT. We hope our results show the potential of LoopLM as a novel scaling direction in the reasoning era. Our model is available here: http://ouro-llm.github.io.

Key facts. Cited by 165 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 62 of its 98 references that are in the map. First posted on arXiv 29 Oct 2025.

Builds on

62 of 98 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025A Survey on Latent Reasoning8 Jul
  2. 2025Llama-Nemotron: Efficient Reasoning Models2 May
  3. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformers24 Feb
  4. 2025Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking19 Feb
  5. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach7 Feb

Other references · 57

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2021Program Synthesis with Large Language Modelscited by 3,637
  7. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  10. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  11. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  12. 2024Qwen2 Technical Reportcited by 2,120
38 more
  1. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  2. 2025Gemma 3 Technical Reportcited by 1,824
  3. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  4. 2020GLU Variants Improve Transformercited by 1,804
  5. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  6. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  7. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  8. 2018Universal Transformerscited by 647
  9. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 559
  10. 2025Humanity's Last Examcited by 491
  11. 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelcited by 335
  12. 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 304
  13. 2022Transformers Learn Shortcuts to Automatacited by 268
  14. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 249
  15. 2019WinoGrande: An Adversarial Winograd Schema Challenge at Scalecited by 204
  16. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithfulcited by 170
  17. 2021PonderNet: Learning to Pondercited by 168
  18. 2024Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Datasetcited by 160
  19. 2023Looped Transformers are Better at Learning Learning Algorithmscited by 149
  20. 2024How to Train Long-Context Language Models (Effectively)cited by 146
  21. 2025Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computationcited by 133
  22. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 113
  23. 2024Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Processcited by 108
  24. 2021Lessons on Parameter Sharing across Layers in Transformerscited by 85
  25. 2024Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspectivecited by 81
  26. 2024Transformers, parallel computation, and logarithmic depthcited by 79
  27. 2025SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplinescited by 79
  28. 2024Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRAcited by 78
  29. 2025Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thoughtcited by 77
  30. 2025AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergycited by 77
  31. 2024Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning?cited by 65
  32. 2025A Little Depth Goes a Long Way: The Expressive Power of Log-Depth Transformerscited by 53
  33. 2025MegaMath: Pushing the Limits of Open Math Corporacited by 51
  34. 2025PonderLM: Pretraining Language Models to Ponder in Continuous Spacecited by 44
  35. 2025Ultra-FineWeb: Efficient Data Filtering and Verification for High-Quality LLM Training Datacited by 38
  36. 2025Learning Compositional Functions with Transformers from Easy-to-Hard Datacited by 36
  37. 2025Understanding Transformer from the Perspective of Associative Memorycited by 34
  38. 2025Transformers Learn to Implement Multi-step Gradient Descent with Chain of Thoughtcited by 32

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 160.

165 papers.

Sort:

By area:

  1. 2026LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulationcited by 61
  2. 2026Parcae: Scaling Laws For Stable Looped Language Modelscited by 55
  3. 2026The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlookcited by 43
  4. 2026Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformerscited by 37
  5. 2026A Mechanistic Analysis of Looped Reasoning Language Modelscited by 33
  6. 2026Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformerscited by 28
  7. 2026How Much Is One Recurrence Worth? Iso-Depth Scaling Laws for Looped Language Modelscited by 24
  8. 2026Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language Modelscited by 21
  9. 2026Recurrent-Depth VLA: Implicit Test-Time Compute Scaling of Vision-Language-Action Models via Latent Iterative Reasoningcited by 18
  10. 2026Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoningcited by 18
  11. 2026SMELT: Scaling Laws for Compute-Matched MoE Looped Transformerscited by 18
  12. 2025Think-at-Hard: Dynamic Looped Transformers for Improved Reasoningcited by 16
38 more
  1. 2026LoopViT: Scaling Visual ARC with Looped Transformerscited by 15
  2. 2026LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Modelscited by 15
  3. 2026Training-Free Looped Transformerscited by 15
  4. 2026Memory-Efficient Looped Transformer: Decoupling Compute from Memory in Looped Language Modelscited by 14
  5. 2026Solve the Loop: Attractor Models for Language and Reasoningcited by 13
  6. 2026Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformerscited by 13
  7. 2026Depth-Recurrent Attention Mixtures: Giving Latent Reasoning the Attention it Deservescited by 12
  8. 2026SpiralFormer: Looped Transformers Can Learn Hierarchical Dependencies via Multi-Resolution Recursioncited by 12
  9. 2026Hyperloop Transformerscited by 12
  10. 2026Adaptive Depth in Looped Transformers: Diagnosing Learned Halting Gates and Trajectory Readoutscited by 12
  11. 2026Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal Structurecited by 11
  12. 2026Adaptive Loops and Memory in Transformers: Think Harder or Know More?cited by 11
  13. 2026Sparse Layers are Critical to Scaling Looped Language Modelscited by 11
  14. 2026Nanbeige4.2-3B: Unlocking Agentic Capabilities in a Compact Modelcited by 11
  15. 2026Residual Context Diffusion Language Modelscited by 10
  16. 2026AdaPonderLM: Gated Pondering Language Models with Token-Wise Adaptive Depthcited by 10
  17. 2026Recursive Multi-Agent Systemscited by 10
  18. 2026LoopMoE: Unifying Iterative Computation with Mixture-of-Experts for Language Modelingcited by 10
  19. 2026On the Residual Scaling of Looped Transformers: Stability and Transferabilitycited by 10
  20. 2026Are Latent Reasoning Models Easily Interpretable?cited by 9
  21. 2026HRM-Text: Efficient Pretraining Beyond Scalingcited by 9
  22. 2026Loop as a Bridge: Can Looped Transformers Truly Link Representation Space and Natural Language Outputs?cited by 8
  23. 2026Prioritize the Process, Not Just the Outcome: Rewarding Latent Thought Trajectories Improves Reasoning in Looped Language Modelscited by 8
  24. 2026ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spacescited by 8
  25. 2026Loop the Loopies!cited by 8
  26. 2026LaDi-RL: Latent Diffusion Reasoning Prevents Entropy Collapse in Reinforcement Learningcited by 7
  27. 2026From Growing to Looping: A Unified View of Iterative Computation in LLMscited by 7
  28. 2026DeepLoop: Depth Scaling for Looped Transformerscited by 7
  29. 2026T^2MLR: Transformer with Temporal Middle-Layer Recurrencecited by 7
  30. 2026Looped Latent Attention: Cross-Loop KV Compression for Looped Transformerscited by 7
  31. 2026Understanding Dynamic Compute Allocation in Recurrent Transformerscited by 6
  32. 2026The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoningcited by 6
  33. 2026LoopQ: Quantization for Looped Language Modelscited by 6
  34. 2026Simply Stabilizing the Loop via Fully Looped Transformercited by 6
  35. 2026Depth-adaptive Inference of Looped Language Models via Continuous Depth Batchingcited by 6
  36. 2026Looped Transformers under the Jacobian Lens: Does the Global Workspace Survive Recurrence?cited by 6
  37. 2026Looping Back to Move Forward: Recursive Transformers for Efficient and Flexible Large Multimodal Modelscited by 5
  38. 2026Step-resolved data attribution for looped transformerscited by 5

50 most cited of 165 All 165 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding
  2. 2025Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughts
  3. 2025Reasoning to Learn from Latent Thoughts
  4. 2025MobileLLM-R1: Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes
  5. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
  6. 2025A Survey of Scaling in Large Language Model Reasoning
  7. 2026Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens
  8. 2025Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Models