mlsanityData to 7 Oct 2026

Critic-CoT: Boosting the reasoning abilities of large language model via Chain-of-thoughts Critic

arXiv 2408.16326 ↗First posted 29 Aug 2024cs.CLcited by 27

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Self-critic has become a crucial mechanism for enhancing the reasoning performance of LLMs. However, current approaches mainly involve basic prompts for intuitive instance-level feedback, which resembles System-1 processes and limits the reasoning capabilities. Moreover, there is a lack of in-depth investigations into the relationship between LLM's ability to criticize and its task-solving performance. To address these issues, we propose Critic-CoT, a novel framework that pushes LLMs toward System-2-like critic capability. Through a step-wise CoT reasoning paradigm and the automatic construction of distant-supervision data without human annotation, Critic-CoT enables LLMs to engage in slow, analytic self-critique and refinement, thereby improving their reasoning abilities. Experiments on GSM8K and MATH demonstrate that our enhanced model significantly boosts task-solving performance by filtering out invalid solutions or iterative refinement. Furthermore, we investigate the intrinsic correlation between critique and task-solving abilities within LLMs, discovering that these abilities can mutually reinforce each other rather than conflict.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 48 of its 68 references that are in the map. First posted on arXiv 29 Aug 2024.

Builds on

48 of 68 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  2. 2024CriticBench: Benchmarking LLMs for Critique-Correct Reasoning22 Feb
  3. 2024Self-Contrast: Better Reflection Through Inconsistent Solving Perspectives4 Jan
  4. 2023Large Language Models Cannot Self-Correct Reasoning Yet3 Oct
  5. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar
  6. 2022STaR: Bootstrapping Reasoning With Reasoning28 Mar

Other references · 42

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  10. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  11. 2023Let's Verify Step by Stepcited by 3,598
  12. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
30 more
  1. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  2. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  3. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  4. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  5. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  6. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  7. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  8. 2024Self-Rewarding Language Modelscited by 654
  9. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
  10. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  11. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  12. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
  13. 2022Self-critiquing models for assisting human evaluatorscited by 372
  14. 2023Language Models can Solve Computer Taskscited by 371
  15. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 293
  16. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 256
  17. 2021ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learningcited by 253
  18. 2024AlphaMath Almost Zero: Process Supervision without Processcited by 160
  19. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 158
  20. 2024Distilling System 2 into System 1cited by 152
  21. 2024InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoningcited by 117
  22. 2023The Generative AI Paradox: "What It Can Create, It May Not Understand"cited by 97
  23. 2023Levels of AGI for Operationalizing Progress on the Path to AGIcited by 94
  24. 2024DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solvingcited by 92
  25. 2023MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluationcited by 57
  26. 2023Improving Large Language Model Fine-tuning for Solving Math Problemscited by 48
  27. 2024LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedbackcited by 40
  28. 2024Large Language Models Can Self-Correct with Key Condition Verificationcited by 39
  29. 2024Towards Scalable Automated Alignment of LLMs: A Surveycited by 33
  30. 2023Critique Ability of Large Language Modelscited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 18; 2026: 6.

27 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
  3. 2025Evaluating LLM-based Agents for Multi-Turn Conversations: A Surveycited by 56
  4. 2025Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning?cited by 43
  5. 2025SPC: Evolving Self-Play Critic via Adversarial Games for LLM Reasoningcited by 40
  6. 2024A Preliminary Study of o1 in Medicine: Are We Closer to an AI Doctor?cited by 33
  7. 2025Table-Critic: A Multi-Agent Framework for Collaborative Criticism and Refinement in Table Reasoningcited by 32
  8. 2024What Makes Large Language Models Reason in (Multi-Turn) Code Generation?cited by 29
  9. 2025Adversarial Reasoning at Jailbreaking Timecited by 27
  10. 2025ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in LLMscited by 23
  11. 2025Boosting LLM Reasoning via Spontaneous Self-Correctioncited by 18
  12. 2025DeepCritic: Deliberate Critique with Large Language Modelscited by 17
15 more
  1. 2025R4ec: A Reasoning, Reflection, and Refinement Framework for Recommendation Systemscited by 16
  2. 2025CodeCriticBench: A Holistic Code Critique Benchmark for Large Language Modelscited by 14
  3. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  4. 2025RealCritic: Towards Effectiveness-Driven Evaluation of Language Model Critiquescited by 12
  5. 2025Temporal Consistency for LLM Reasoning Process Error Identificationcited by 10
  6. 2026Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learningcited by 8
  7. 2025R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoningcited by 7
  8. 2025Critique-RL: Training Language Models for Critiquing through Two-Stage Reinforcement Learningcited by 3
  9. 2026M^4olGen: Multi-Agent, Multi-Stage Molecular Generation under Precise Multi-Property Constraintscited by 3
  10. 2026Verifiable Reasoning for LLM-based Generative Recommendationcited by 2
  11. 2026Simple Role Assignment is Extraordinarily Effective for Safety Alignmentcited by 1
  12. 2026Agentic AI for Commercial Insurance Underwriting with Adversarial Self-Critiquecited by 1
  13. 2025Audit-of-Understanding: Posterior-Constrained Inference for Mathematical Reasoning in Language Modelscited by 0
  14. 2025Separate the Wheat from the Chaff: Winnowing Down Divergent Views in Retrieval Augmented Generationcited by 0
  15. 2026Learning from Failures: Correction-Oriented Policy Optimization with Verifiable Rewardscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
  2. 2025When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMs
  3. 2025Dancing with Critiques: Enhancing LLM Reasoning with Stepwise Natural Language Self-Critique
  4. 2025Double-Checker: Enhancing Reasoning of Slow-Thinking LLMs via Self-Critical Fine-Tuning
  5. 2025Short-Path Prompting in LLMs: Analyzing Reasoning Instability and Solutions for Robust Performance
  6. 2025Stepwise Think-Critique: Interleaved Reasoning and Self-Critique in a Single LLM
  7. 2024Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
  8. 2025The Challenge of Teaching Reasoning to LLMs Without RL or Distillation