mlsanityData to 14 Oct 2026

Stepwise Think-Critique: Interleaved Reasoning and Self-Critique in a Single LLM

arXiv 2512.15662 ↗First posted 17 Dec 2025cs.AIcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Human beings solve complex problems through critical thinking, where reasoning and evaluation are intertwined to converge toward correct solutions. However, most existing large language models (LLMs) treat the reasoning and verification as separate processes: they either generate reasoning without explicit self-checking or rely on external verifiers to detect errors post hoc. The former lacks immediate feedback, while the latter increases system complexity and hinders synchronized learning. Motivated by human critical thinking, we propose Stepwise Think-Critique (STC), an end-to-end trainable framework in which a single LLM emits a structured, step-level critique inline with each reasoning step. STC is trained with reinforcement learning that complements reasoning rewards with a critique-consistency reward derived from final-answer correctness, jointly optimizing reasoning correctness and critique reliability. On five mathematical reasoning benchmarks, STC improves Pass@1 by 7.2% over the 1.5B base model and attains 67.4% step-level critique F1, surpassing seven external process reward models evaluated at their per-dataset oracle thresholds---a step toward LLMs with built-in critical thinking.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 16 of its 28 references that are in the map. First posted on arXiv 17 Dec 2025.

Builds on

16 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Training Language Models to Self-Correct via Reinforcement Learning19 Sep

Other references · 15

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  3. 2025Process Reinforcement through Implicit Rewardscited by 504
  4. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 480
  5. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 415
  6. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  7. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Taskscited by 285
  8. 2025REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalizationcited by 241
  9. 2025Process Reward Models That Thinkcited by 118
  10. 2025GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoningcited by 97
  11. 2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53
  12. 2025ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMscited by 46
3 more
  1. 2025RL Tango: Reinforcing Generator and Verifier Together for Language Reasoningcited by 40
  2. 2025StepWiser: Stepwise Generative Judges for Wiser Reasoningcited by 28
  3. 2025Error Typing for Smarter Rewards: Improving Process Reward Models with Error-Aware Hierarchical Supervisioncited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Dancing with Critiques: Enhancing LLM Reasoning with Stepwise Natural Language Self-Critique
  2. 2024CriticBench: Benchmarking LLMs for Critique-Correct Reasoning
  3. 2024Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
  4. 2025Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback
  5. 2025Large Reasoning Models are not thinking straight: on the unreliability of thinking trajectories
  6. 2026ThinkTwice: Jointly Optimizing Large Language Models for Reasoning and Self-Refinement
  7. 2025Critique-Coder: Enhancing Coder Models by Critique Reinforcement Learning
  8. 2024Critic-CoT: Boosting the reasoning abilities of large language model via Chain-of-thoughts Critic