mlsanityData to 14 Oct 2026

To CoT or To Loop? A Formal Comparison Between Chain-of-Thought and Looped Transformers

arXiv 2505.19245 ↗First posted 25 May 2025cs.LG cs.AIcited by 11

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Chain-of-Thought (CoT) and Looped Transformers have been shown to empirically improve performance on reasoning tasks and to theoretically enhance expressivity by recursively increasing the number of computational steps. However, their comparative capabilities are still not well understood. In this paper, we provide a formal analysis of their respective strengths and limitations. We show that Looped Transformers can efficiently simulate parallel computations for deterministic tasks, which we formalize as evaluation over directed acyclic graphs. In contrast, CoT with stochastic decoding excels at approximate inference for compositional structures, namely self-reducible problems. These separations suggest the tasks for which depth-driven recursion is more suitable, thereby offering practical cues for choosing between reasoning paradigms.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 18 of its 30 references that are in the map. First posted on arXiv 25 May 2025.

Builds on

18 of 30 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoning20 Jun
  2. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problems20 Feb
  3. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective24 May
  4. 2023Why think step by step? Reasoning emerges from the locality of experience7 Apr
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 12

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  3. 2018Universal Transformerscited by 647
  4. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Modelscited by 290
  5. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 240
  6. 2023Looped Transformers as Programmable Computerscited by 237
  7. 2022The Parallelism Tradeoff: Limitations of Log-Precision Transformerscited by 193
  8. 2024From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Modelscited by 137
  9. 2024Transformers, parallel computation, and logarithmic depthcited by 79
  10. 2024MoEUT: Mixture-of-Experts Universal Transformerscited by 59
  11. 2025A Little Depth Goes a Long Way: The Expressive Power of Log-Depth Transformerscited by 53
  12. 2024On Expressive Power of Looped Transformers: Theoretical Analysis and Enhancement via Timestep Encodingcited by 36

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 6.

11 papers.

  1. 2025SIM-CoT: Supervised Implicit Chain-of-Thoughtcited by 70
  2. 2026LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulationcited by 61
  3. 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 43
  4. 2026Training-Free Looped Transformerscited by 15
  5. 2026Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformerscited by 13
  6. 2025MeSH: Memory-as-State-Highways for Recursive Transformerscited by 8
  7. 2026Loop the Loopies!cited by 8
  8. 2025On the Reasoning Abilities of Masked Diffusion Language Modelscited by 7
  9. 2025DAG-Math: Graph-of-Thought Guided Mathematical Reasoning in LLMscited by 5
  10. 2026CoLT: Reasoning with Chain of Latent Tool Callscited by 1
  11. 2026Running Fewer Loops Removes Refusal Behaviour in Recurrent-Depth Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Barriers to Universal Reasoning With Transformers (And How to Overcome Them)
  2. 2025The Kinetics of Reasoning: How Chain-of-Thought Shapes Learning in Transformers?
  3. 2025Analyzing the Power of Chain of Thought through Memorization Capabilities
  4. 2025Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought
  5. 2025Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer
  6. 2024Iteration Head: A Mechanistic Study of Chain-of-Thought
  7. 2026Chain-of-Thought and Compressed Looped Transformers: A Memory-Budget Separation
  8. 2025Continuous Chain of Thought Enables Parallel Exploration and Reasoning