mlsanityData to 7 Oct 2026

Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought

arXiv 2501.04682 ↗First posted 8 Jan 2025cs.AI cs.CLcited by 108

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

We propose a novel framework, Meta Chain-of-Thought (Meta-CoT), which extends traditional Chain-of-Thought (CoT) by explicitly modeling the underlying reasoning required to arrive at a particular CoT. We present empirical evidence from state-of-the-art models exhibiting behaviors consistent with in-context search, and explore methods for producing Meta-CoT via process supervision, synthetic data generation, and search algorithms. Finally, we outline a concrete pipeline for training a model to produce Meta-CoTs, incorporating instruction tuning with linearized search traces and reinforcement learning post-training. Finally, we discuss open research questions, including scaling laws, verifier roles, and the potential for discovering novel reasoning algorithms. This work provides a theoretical and practical roadmap to enable Meta-CoT in LLMs, paving the way for more powerful and human-like reasoning in artificial intelligence.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 75 of its 108 references that are in the map. First posted on arXiv 8 Jan 2025.

Builds on

75 of 108 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoning20 Jun
  2. 2024Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrapping21 Feb
  3. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problems20 Feb
  4. 2023Training Chain-of-Thought via Latent-Variable Inference28 Nov
  5. 2023The Expressive Power of Transformers with Chain of Thought11 Oct
  6. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks22 Nov

Other references · 69

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  7. 2013Auto-Encoding Variational Bayescited by 3,980
  8. 2023Let's Verify Step by Stepcited by 3,598
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  10. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  12. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
38 more
  1. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  2. 2016Bag of Tricks for Efficient Text Classificationcited by 869
  3. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  4. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  5. 2018Universal Transformerscited by 613
  6. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 564
  7. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  8. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 533
  9. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  10. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
  11. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
  12. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
  13. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  14. 2016FastText.zip: Compressing text classification modelscited by 331
  15. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 325
  16. 2023SemDeDup: Data-efficient learning at web-scale through semantic deduplicationcited by 295
  17. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  18. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Modelscited by 277
  19. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 267
  20. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 265
  21. 2023The False Promise of Imitating Proprietary LLMscited by 254
  22. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 238
  23. 2024A Survey on Data Selection for Language Modelscited by 230
  24. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
  25. 2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agentscited by 201
  26. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 194
  27. 2024Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Datacited by 184
  28. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 168
  29. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improvecited by 162
  30. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 158
  31. 2024Orca-Math: Unlocking the potential of SLMs in Grade School Mathcited by 151
  32. 2024Tree Search for Language Model Agentscited by 148
  33. 2024Generative Reward Modelscited by 148
  34. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 146
  35. 2024Stream of Search (SoS): Learning to Search in Languagecited by 141
  36. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizingcited by 123
  37. 2024Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Processcited by 107
  38. 2024Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMscited by 103

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 87; 2026: 21.

108 papers.

Sort:

By area:

  1. 2025s1: Simple test-time scalingcited by 1,311
  2. 2025LIMO: Less is More for Reasoningcited by 488
  3. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  4. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 373
  5. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  6. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  7. 2025Scalable Best-of-N Selection for Large Language Models via Self-Certaintycited by 182
  8. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 146
  9. 2025Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuningcited by 121
  10. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoningcited by 118
  11. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 106
  12. 2025Agentic Large Language Models, a surveycited by 105
38 more
  1. 2025Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Modelscited by 104
  2. 2025DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Processcited by 97
  3. 2025DeepSeek-R1 Thoughtology: Let's think about LLM Reasoningcited by 91
  4. 2025Scaling Test-Time Compute Without Verification or RL is Suboptimalcited by 82
  5. 2025Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chainscited by 81
  6. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 78
  7. 2025Persona Features Control Emergent Misalignmentcited by 76
  8. 2025All Roads Lead to Likelihood: The Value of Reinforcement Learning in Fine-Tuningcited by 58
  9. 2025CodeI/O: Condensing Reasoning Patterns via Code Input-Output Predictioncited by 54
  10. 2025Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learningcited by 51
  11. 2025Crosslingual Reasoning through Test-Time Scalingcited by 46
  12. 2025REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websitescited by 43
  13. 2025Monte Carlo Tree Diffusion for System 2 Planningcited by 33
  14. 2025Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoningcited by 33
  15. 2025Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoningcited by 32
  16. 2025Beyond Verifiable Rewards: Scaling Reinforcement Learning for Language Models to Unverifiable Datacited by 29
  17. 2025Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Propertiescited by 28
  18. 2025Adversarial Reasoning at Jailbreaking Timecited by 27
  19. 2025Optimizing Anytime Reasoning via Budget Relative Policy Optimizationcited by 27
  20. 2025Metastable Dynamics of Chain-of-Thought Reasoning: Provable Benefits of Search, RL and Distillationcited by 25
  21. 2025On the Emergence of Thinking in LLMs I: Searching for the Right Intuitioncited by 22
  22. 2026Render-of-Thought: Rendering Textual Chain-of-Thought as Images for Visual Latent Reasoningcited by 19
  23. 2025On the Effect of Sampling Diversity in Scaling LLM Inferencecited by 18
  24. 2025Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPOcited by 18
  25. 2025Boosting LLM Reasoning via Spontaneous Self-Correctioncited by 18
  26. 2025Learning When to Plan: Efficiently Allocating Test-Time Compute for LLM Agentscited by 18
  27. 2025Forgotten Polygons: Multimodal Large Language Models are Shape-Blindcited by 16
  28. 2025Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoningcited by 16
  29. 2025QFFT, Question-Free Fine-Tuning for Adaptive Reasoningcited by 15
  30. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
  31. 2025Reinforcement Learning vs. Distillation: Understanding Accuracy and Capability in LLM Reasoningcited by 14
  32. 2025Have Large Language Models Learned to Reason? A Characterization via 3-SAT Phase Transitioncited by 13
  33. 2025Chain-of-Thought for Autonomous Driving: A Comprehensive Survey and Future Prospectscited by 12
  34. 2025Let Me Think! A Long Chain-of-Thought Can Be Worth Exponentially Many Short Onescited by 12
  35. 2025Improving Pre-Trained Vision-Language-Action Policies with Model-Based Searchcited by 12
  36. 2025Investigating Inference-time Scaling for Chain of Multi-modal Thought: A Preliminary Studycited by 11
  37. 2025Don't Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Modelscited by 11
  38. 2025Through the Valley: Path to Effective Long CoT Training for Small Language Modelscited by 10

50 most cited of 108 All 108 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Rethinking Chain-of-Thought from the Perspective of Self-Training
  2. 2024Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
  3. 2025From Reasoning to Super-Intelligence: A Search-Theoretic Perspective
  4. 2024MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMs
  5. 2024Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs
  6. 2024Understanding Chain-of-Thought in LLMs through Information Theory
  7. 2026Hierarchical Chain-of-Thought: Enhancing LLM Reasoning Performance and Efficiency
  8. 2024Strategic Chain-of-Thought: Guiding Accurate Reasoning in LLMs through Strategy Elicitation