Reasoning Steps as Curriculum: Using Depth of Thought as a Difficulty Signal for Tuning LLMs
On the map: LLM reasoning → Prompting and chain-of-thought reasoning
Abstract (arXiv)
Curriculum learning for training LLMs requires a difficulty signal that aligns with reasoning while remaining scalable and interpretable. We propose a simple premise: tasks that demand deeper depth of thought for humans should also be harder for models. Accordingly, we define difficulty as depth of thought (DoT) and operationalize it by counting the discrete steps in a teacher model's reasoning trace (e.g., Chain-of-Thought). We then train with a shallow to deep curriculum ordered by this DoT and outline how to derive, validate, and schedule it at scale. Our position yields three testable hypotheses: (i) DoT correlates with conventional difficulty on reasoning benchmarks, (ii) DoT-ordered curricula outperform length- or judge-scored curricula under matched budgets, and (iii) the difficulty is robust across teacher models given light formatting controls. We propose an evaluation framework and discuss threats to validity (teacher style, length confounds) alongside practical mitigations. Taken together, we aim to move toward cognitively grounded, interpretable curricula for reasoning-centric training.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 6 of its 8 references that are in the map. First posted on arXiv 13 Aug 2025.
Builds on
6 of 8 references.
Closest ancestors · 2
Same direction, newest first
- 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step23 May
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan
Other references · 4
- 2024Let's Learn Step by Step: Enhancing In-Context Learning Ability with Curriculum Learningcited by 47
- 2025Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex Reasoningcited by 12
- 2025Long Is More Important Than Difficult for Training Reasoning Modelscited by 11
- 2024Phased Instruction Fine-Tuning for Large Language Modelscited by 2
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Rethinking Easy-to-Hard: Limits of Curriculum Learning in Post-Training for Deductive Reasoning
- 2025Can an Easy-to-Hard Curriculum Make Reasoning Emerge in Small Language Models? Evidence from a Four-Stage Curriculum on GPT-2
- 2024Automatic Curriculum Expert Iteration for Reliable LLM Reasoning
- 2025Short-Path Prompting in LLMs: Analyzing Reasoning Instability and Solutions for Robust Performance
- 2024Can LLMs Learn by Teaching for Better Reasoning? A Preliminary Study
- 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning
- 2024Beyond Imitation: Learning Key Reasoning Steps from Dual Chain-of-Thoughts in Reasoning Distillation
- 2025Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Prompting