mlsanityData to 7 Oct 2026

Ladders of Thought: A Self-Evolving Curriculum of Progressively Simplified Reasoning Traces

arXiv 2609.25643 ↗First posted 22 Sep 2026cs.AI cs.LGcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking also fits: Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

Large language models (LLMs) excel at reasoning when scaled to hundreds of billions of parameters, but small- and mid-scale models remain brittle reasoners even with knowledge distillation (KD). We present Ladders-of-Thought (LoT), a framework that improves reasoning by combining progressive question rewrites with a self-evolving curriculum. LoT automatically generates semantically faithful but easier variants of reasoning problems, organizes them into difficulty buckets using step-based measures, and employs a self-evolving bandit scheduler to allocate training adaptively. Evaluated on two reasoning domains, math and multi-hop reasoning, across 1-8B models from different families, LoT consistently improves over KD. It delivers large gains on arithmetic tasks (e.g., +32 percentage points on AddSub, +25pp on SVAMP), +2-8pp improvements on in-domain test splits, and strong though dataset-dependent benefits on multi-hop reasoning (e.g., +16pp on QASC, +25pp on StrategyQA). LoT also converges faster than staged curricula, highlighting the value of adaptive progression. These results show that progressive rewrites coupled with adaptive curricula provide a simple yet effective recipe for strengthening reasoning in smaller LLMs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 53 of its 57 references that are in the map. First posted on arXiv 22 Sep 2026.

Builds on

53 of 57 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoning26 Apr
  2. 2023Making Large Language Models Better Reasoners with Alignment5 Sep
  3. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  4. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar

Other references · 49

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  8. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  9. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  10. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  11. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  12. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
37 more
  1. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  2. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  3. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  4. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  5. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  6. 2022PAL: Program-aided Language Modelscited by 635
  7. 2016Solving General Arithmetic Word Problemscited by 353
  8. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
  9. 2022Large Language Models Are Reasoning Teacherscited by 318
  10. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  11. 2022Teaching Small Language Models to Reasoncited by 279
  12. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  13. 2021Symbolic Knowledge Distillation: from General Language Models to Commonsense Modelscited by 270
  14. 2021Curriculum Learning: A Surveycited by 257
  15. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 249
  16. 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 248
  17. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 231
  18. 2017Teacher-Student Curriculum Learningcited by 197
  19. 2023Implicit Chain of Thought Reasoning via Knowledge Distillationcited by 165
  20. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 157
  21. 2024The Impact of Reasoning Step Length on Large Language Modelscited by 152
  22. 2021Explaining Answers with Entailment Treescited by 148
  23. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvementcited by 144
  24. 2025Small Models Struggle to Learn from Strong Reasonerscited by 109
  25. 2025Self-Evolving Curriculum for LLM Reasoningcited by 107
  26. 2023SCOTT: Self-Consistent Chain-of-Thought Distillationcited by 103
  27. 2023Cumulative Reasoning with Large Language Modelscited by 94
  28. 2025Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoningcited by 89
  29. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learningcited by 88
  30. 2023TinyGSM: achieving >80% on GSM8k with small language modelscited by 68
  31. 2023Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Taskscited by 62
  32. 2024Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Applicationcited by 55
  33. 2024Let's Learn Step by Step: Enhancing In-Context Learning Ability with Curriculum Learningcited by 47
  34. 2023Dialogue Chain-of-Thought Distillation for Commonsense-aware Conversational Agentscited by 41
  35. 2024Achieving >97% on GSM8K: Deeply Understanding the Problems Makes LLMs Better Solvers for Math Word Problemscited by 31
  36. 2025LADDER: Self-Improving LLMs Through Recursive Problem Decompositioncited by 24
  37. 2024Divide-or-Conquer? Which Part Should You Distill Your LLM?cited by 17

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Questioning the Questions: Sustaining Self-Evolution in Reasoning Models
  2. 2026TTCS: Test-Time Curriculum Synthesis for Self-Evolving
  3. 2025Climbing the Ladder of Reasoning: What LLMs Can-and Still Can't-Solve after SFT?
  4. 2025A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning
  5. 2026Preventing Curriculum Collapse in Self-Evolving Reasoning Systems
  6. 2025Socratic-Zero : Bootstrapping Reasoning via Data-Free Agent Co-evolution
  7. 2025From Efficiency to Adaptivity: A Deeper Look at Adaptive Reasoning in Large Language Models
  8. 2026Knowing When Thinking Is Not Enough: Teaching Small Reasoning Models to Reason Beyond Their Parametric Knowledge