mlsanityData to 7 Oct 2026

Rethinking Chain-of-Thought from the Perspective of Self-Training

arXiv 2412.10827 ↗First posted 14 Dec 2024cs.CL cs.AIcited by 10

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

Chain-of-thought (CoT) reasoning has emerged as an effective approach for activating latent capabilities in LLMs. Interestingly, we observe that both CoT reasoning and self-training share the core objective: iteratively leveraging model-generated information to progressively reduce prediction uncertainty. Building on this insight, we propose a novel CoT framework to improve reasoning performance. Our framework integrates two key components: (i) a task-specific prompt module that optimizes the initial reasoning process, and (ii) an adaptive reasoning iteration module that dynamically refines the reasoning process and addresses the limitations of previous CoT approaches, ie over-reasoning and high similarity between consecutive reasoning iterations. Extensive experiments demonstrate that the proposed method achieves significant advantages in both performance and computational efficiency.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 39 references that are in the map. First posted on arXiv 14 Dec 2024.

Builds on

26 of 39 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Contrastive Chain-of-Thought Prompting15 Nov
  2. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future27 Sep
  3. 2023Measuring Faithfulness in Chain-of-Thought Reasoning17 Jul
  4. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models6 May
  5. 2022Automatic Chain of Thought Prompting in Large Language Models7 Oct
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 19

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  3. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  4. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  5. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  6. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  7. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 537
  8. 2019Confidence Regularized Self-Trainingcited by 418
  9. 2016Solving General Arithmetic Word Problemscited by 353
  10. 2021A Survey on Deep Semi-supervised Learningcited by 250
  11. 2024Evaluation of OpenAI o1: Opportunities and Challenges of AGIcited by 127
  12. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 122
7 more
  1. 2024Concise Thoughts: Impact of Output Length on LLM Reasoning and Costcited by 108
  2. 2022Self-Training: A Surveycited by 85
  3. 2022Debiased Self-Training for Semi-Supervised Learningcited by 62
  4. 2023Re-Reading Improves Reasoning in Large Language Modelscited by 37
  5. 2024How Much Can RAG Help the Reasoning of LLM?cited by 31
  6. 2021Self-training Converts Weak Learners to Strong Learners in Mixture Modelscited by 19
  7. 2024Constrained Reasoning Chains for Enhancing Theory-of-Mind in Large Language Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 5.

10 papers.

  1. 2025Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learningcited by 32
  2. 2025Hawkeye:Efficient Reasoning with Model Collaborationcited by 11
  3. 2025Is Depth All You Need? An Exploration of Iterative Reasoning in LLMscited by 6
  4. 2026Unveiling the Entropy Dynamics of Chain-of-Thought Reasoningcited by 4
  5. 2026Revisiting Chain-of-Thought Reasoning under Limited Supervision: Semi-supervised Chain-of-Thought Learningcited by 4
  6. 2026DemMA: Dementia Multi-Turn Dialogue Agent with Expert-Guided Reasoning and Action Simulationcited by 2
  7. 2025Three tiers of computation in transformers and in brain architecturescited by 1
  8. 2025GEM: Generative Entropy-Guided Preference Modeling for Few-shot Alignment of LLMscited by 0
  9. 2026A Training-Free Regeneration Paradigm: Contrastive Reflection Memory Guided Self-Verification and Self-Improvementcited by 0
  10. 2026PRISMA: Preference-Reinforced Self-Training Approach for Interpretable Emotionally Intelligent Negotiation Dialoguescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
  2. 2026Reinforcing Chain-of-Thought Reasoning with Self-Evolving Rubrics
  3. 2026S3-CoT: Self-Sampled Succinct Reasoning Enables Efficient Chain-of-Thought LLMs
  4. 2024Strategic Chain-of-Thought: Guiding Accurate Reasoning in LLMs through Strategy Elicitation
  5. 2024Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
  6. 2024Understanding Chain-of-Thought in LLMs through Information Theory
  7. 2026Hierarchical Chain-of-Thought: Enhancing LLM Reasoning Performance and Efficiency
  8. 2025From Reasoning to Super-Intelligence: A Search-Theoretic Perspective