mlsanityData to 7 Oct 2026

LADDER: Self-Improving LLMs Through Recursive Problem Decomposition

arXiv 2503.00735 ↗First posted 2 Mar 2025cs.LG cs.AIcited by 24

On the map: Synthetic data and self-improvement for LLMs → Synthetic reasoning data and bootstrapping

Abstract (arXiv)

We introduce LADDER (Learning through Autonomous Difficulty-Driven Example Recursion), a framework which enables Large Language Models to autonomously improve their problem-solving capabilities through self-guided learning by recursively generating and solving progressively simpler variants of complex problems. Unlike prior approaches that require curated datasets or human feedback, LADDER leverages a model's own capabilities to generate easier question variants. We demonstrate LADDER's effectiveness in the subject of mathematical integration, improving Llama 3.2 3B's accuracy from 1% to 82% on undergraduate-level problems and enabling Qwen2.5 7B Deepseek-R1 Distilled to achieve 73% on the MIT Integration Bee qualifying examination. We also introduce TTRL (Test-Time Reinforcement Learning), where we perform reinforcement learning on variants of test problems at inference time. TTRL enables Qwen2.5 7B Deepseek-R1 Distilled to achieve a state-of-the-art score of 90% on the MIT Integration Bee qualifying examination, surpassing OpenAI o1's performance. These results show how self-directed strategic learning can achieve significant capability improvements without relying on architectural scaling or human supervision.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Synthetic reasoning data and bootstrapping, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 6 of its 9 references that are in the map. First posted on arXiv 2 Mar 2025.

Builds on

6 of 9 references.

References · 6

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  5. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
  6. 2022On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgettingcited by 80

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 12; 2026: 12.

24 papers.

Sort:

By area:

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  2. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 162
  3. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  4. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  5. 2025Learning to Reason for Long-Form Story Generationcited by 34
  6. 2025RLSR: Reinforcement Learning from Self Rewardcited by 14
  7. 2025DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learningcited by 11
  8. 2025Local Mixtures of Experts: Essentially Free Test-Time Training via Model Mergingcited by 9
  9. 2025Test-time Scaling Techniques in Theoretical Physics -- A Comparison of Methods on the TPBench Datasetcited by 7
  10. 2025Achieving Olympiad-Level Geometry Large Language Model Agent via Complexity Boosting Reinforcement Learningcited by 6
  11. 2025A quantum semantic framework for natural language processingcited by 5
  12. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlookcited by 5
12 more
  1. 2026Detecting and Mitigating the Correct-Answer Extinction Window in Test-Time Reinforcement Learning with Majority Votingcited by 5
  2. 2026Training LLMs for Divide-and-Conquer Reasoning Elevates Test-Time Scalabilitycited by 4
  3. 2026Recursive Experiential-Working Memory Evolution for Long-Horizon Agent Harnessescited by 4
  4. 2025When Your Own Output Becomes Your Training Data: Noise-to-Meaning Loops and a Formal RSI Triggercited by 1
  5. 2026Amplification Effects in Test-Time Reinforcement Learning: Safety and Reasoning Vulnerabilitiescited by 1
  6. 2025Knowledge Conceptualization Impacts RAG Efficacycited by 0
  7. 2026ReLAPSe: Reinforcement-Learning-trained Adversarial Prompt Search for Erased concepts in unlearned diffusion modelscited by 0
  8. 2026VERIFY-RL: Verifiable Recursive Decomposition for Reinforcement Learning in Mathematical Reasoningcited by 0
  9. 2026RVLM: Recursive Vision-Language Models with Adaptive Depthcited by 0
  10. 2026Be Your Own Teacher: Steering Protein Language Models via Unsupervised Reward Optimizationcited by 0
  11. 2026SCOPE-RL: Optimizing Reasoning Paths Before and After Successcited by 0
  12. 2026Ladders of Thought: A Self-Evolving Curriculum of Progressively Simplified Reasoning Tracescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning Like Humans: Advancing LLM Reasoning Capabilities via Adaptive Difficulty Curriculum Learning and Expert-Guided Self-Reformulation
  2. 2024Can Large Language Models Invent Algorithms to Improve Themselves?: Algorithm Discovery for Recursive Self-Improvement through Reinforcement Learning
  3. 2026Test-time Recursive Thinking: Self-Improvement without External Feedback
  4. 2025Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learning
  5. 2025LogicPuzzleRL: Cultivating Robust Mathematical Reasoning in LLMs via Reinforcement Learning
  6. 2026Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability
  7. 2026TTCS: Test-Time Curriculum Synthesis for Self-Evolving
  8. 2025SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning