mlsanityData to 7 Oct 2026

Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning?

arXiv 2502.19361 ↗First posted 26 Feb 2025cs.CLcited by 43

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Recently, o1-like models have drawn significant attention, where these models produce the long Chain-of-Thought (CoT) reasoning steps to improve the reasoning abilities of existing Large Language Models (LLMs). In this paper, to understand the qualities of these long CoTs and measure the critique abilities of existing LLMs on these long CoTs, we introduce the DeltaBench, including the generated long CoTs from different o1-like models (e.g., QwQ, DeepSeek-R1) for different reasoning tasks (e.g., Math, Code, General Reasoning), to measure the ability to detect errors in long CoT reasoning. Based on DeltaBench, we first perform fine-grained analysis of the generated long CoTs to discover the effectiveness and efficiency of different o1-like models. Then, we conduct extensive evaluations of existing process reward models (PRMs) and critic models to detect the errors of each annotated process, which aims to investigate the boundaries and limitations of existing PRMs and critic models. Finally, we hope that DeltaBench could guide developers to better understand the long CoT reasoning abilities of their models.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 58 references that are in the map. First posted on arXiv 26 Feb 2025.

Builds on

20 of 58 references.

References · 20

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  6. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  7. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  8. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  9. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  10. 2024AlphaMath Almost Zero: Process Supervision without Processcited by 160
  11. 2024On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Taskscited by 137
  12. 2024CriticBench: Benchmarking LLMs for Critique-Correct Reasoningcited by 88
8 more
  1. 2023LLMs cannot find reasoning errors, but can correct them given the error locationcited by 85
  2. 2024MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Seriescited by 67
  3. 2024GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Modelscited by 42
  4. 2024Large Language Models Can Self-Correct with Key Condition Verificationcited by 39
  5. 2024Chinese SimpleQA: A Chinese Factuality Evaluation for Large Language Modelscited by 39
  6. 2024Critic-CoT: Boosting the reasoning abilities of large language model via Chain-of-thoughts Criticcited by 27
  7. 2023Critique Ability of Large Language Modelscited by 20
  8. 2025AIR: Complex Instruction Generation via Automatic Iterative Refinementcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 32; 2026: 11.

43 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMscited by 54
  3. 2025xVerify: Efficient Answer Verifier for Reasoning Model Evaluationscited by 53
  4. 2025Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspectivecited by 32
  5. 2025Synergizing RAG and Reasoning: A Systematic Reviewcited by 27
  6. 2025Measuring the Faithfulness of Thinking Drafts in Large Reasoning Modelscited by 22
  7. 2025What Makes a Good Reasoning Chain? Uncovering Structural Patterns in Long Chain-of-Thought Reasoningcited by 21
  8. 2025CoRT: Code-integrated Reasoning within Thinkingcited by 15
  9. 2025CriticLean: Critic-Guided Reinforcement Learning for Mathematical Formalizationcited by 15
  10. 2025Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systemscited by 13
  11. 2025WebDevJudge: Evaluating (M)LLMs as Critiques for Web Development Qualitycited by 10
  12. 2025SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Modelscited by 10
31 more
  1. 2025Process or Result? Manipulated Ending Tokens Can Mislead Reasoning LLMs to Ignore the Correct Reasoning Stepscited by 9
  2. 2025Read Before You Think: Mitigating LLM Comprehension Failures with Step-by-Step Readingcited by 8
  3. 2025Graph of Verification: Structured Verification of LLM Reasoning with Directed Acyclic Graphscited by 8
  4. 2025DNR Bench: Benchmarking Over-Reasoning in Reasoning LLMscited by 7
  5. 2025Reasoning or Not? A Comprehensive Evaluation of Reasoning LLMs for Dialogue Summarizationcited by 6
  6. 2025RefCritic: Training Long Chain-of-Thought Critic Models with Refinement Feedbackcited by 6
  7. 2025R1-ACT: Efficient Reasoning Model Safety Alignment by Activating Safety Knowledgecited by 6
  8. 2025Meta-R1: Empowering Large Reasoning Models with Metacognitioncited by 6
  9. 2025TraPO: A Semi-Supervised Reinforcement Learning Framework for Boosting LLM Reasoningcited by 6
  10. 2026Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectoriescited by 6
  11. 2025Off-Trajectory Reasoning: Can LLMs Collaborate on Reasoning Trajectory?cited by 5
  12. 2025Teaching Language Models to Reason with Toolscited by 5
  13. 2026Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalizationcited by 5
  14. 2025Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluationcited by 4
  15. 2025Self-Verifying Reflection Helps Transformers with CoT Reasoningcited by 4
  16. 2026Can LLMs Compress (and Decompress)? Evaluating Code Understanding and Execution via Invertibilitycited by 4
  17. 2025Can Large Multimodal Models Actively Recognize Faulty Inputs? A Systematic Evaluation Framework of Their Input Scrutiny Abilitycited by 3
  18. 2025LATTS: Locally Adaptive Test-Time Scalingcited by 3
  19. 2025Stop Before You Fail: Operational Capability Boundaries for Mitigating Unproductive Reasoning in Large Reasoning Modelscited by 3
  20. 2026Long-form RewardBench: Evaluating Reward Models for Long-form Generationcited by 2
  21. 2025NCV: A Node-Wise Consistency Verification Approach for Low-Cost Structured Error Localization in LLM Reasoningcited by 1
  22. 2026Decomposing the Delta: What Do Models Actually Learn from Preference Pairs?cited by 1
  23. 2026ReasoningLens: Hierarchical Visualization and Diagnostic Auditing for Large Reasoning Modelscited by 1
  24. 2025Stepwise Reasoning Checkpoint Analysis: A Test Time Scaling Method to Enhance LLMs' Reasoningcited by 0
  25. 2025Can Large Language Models Reason About Complex Execution Paths? An Empirical Study on Pythoncited by 0
  26. 2025On Finding Inconsistencies in Documentscited by 0
  27. 2026Beyond Model Scaling: Test-Time Intervention for Efficient Deep Reasoningcited by 0
  28. 2026Distilling Long-CoT Reasoning through Collaborative Step-wise Multi-Teacher Decodingcited by 0
  29. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modescited by 0
  30. 2026LCoT-GV: Graph Attention Networks for Verifying Long Reasoning Chains in Large Language Modelscited by 0
  31. 2026The Error You See Is Not the Error You Made: Progression-aware Reasoning Origin for Reasoning Error Localizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillation
  2. 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!
  3. 2024Understanding Chain-of-Thought in LLMs through Information Theory
  4. 2025Long Chain-of-Thought Reasoning Across Languages
  5. 2025Efficient Long CoT Reasoning in Small Language Models
  6. 2025Early Stopping Chain-of-thoughts in Large Language Models
  7. 2024The Impact of Reasoning Step Length on Large Language Models
  8. 2025Demystifying Long Chain-of-Thought Reasoning in LLMs