mlsanityData to 14 Oct 2026

Learning to Plan Before Answering: Self-Teaching LLMs to Learn Abstract Plans for Problem Solving

arXiv 2505.00031 ↗First posted 28 Apr 2025cs.CL cs.AIcited by 4

On the map: Synthetic data and self-improvement for LLMs → Synthetic reasoning data and bootstrapping

Abstract (arXiv)

In the field of large language model (LLM) post-training, the effectiveness of utilizing synthetic data generated by the LLM itself has been well-presented. However, a key question remains unaddressed: what essential information should such self-generated data encapsulate? Existing approaches only produce step-by-step problem solutions, and fail to capture the abstract meta-knowledge necessary for generalization across similar problems. Drawing insights from cognitive science, where humans employ high-level abstraction to simplify complex problems before delving into specifics, we introduce a novel self-training algorithm: LEarning to Plan before Answering (LEPA). LEPA trains the LLM to formulate anticipatory plans, which serve as abstract meta-knowledge for problem-solving, before engaging with the intricacies of problems. This approach not only outlines the solution generation path but also shields the LLM from the distraction of irrelevant details. During data generation, LEPA first crafts an anticipatory plan based on the problem, and then generates a solution that aligns with both the plan and the problem. LEPA refines the plan through self-reflection, aiming to acquire plans that are instrumental in yielding correct solutions. During model optimization, the LLM is trained to predict both the refined plans and the corresponding solutions. By efficiently extracting and utilizing the anticipatory plans, LEPA demonstrates remarkable superiority over conventional algorithms on various challenging natural language reasoning benchmarks.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Synthetic reasoning data and bootstrapping, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 41 of its 49 references that are in the map. First posted on arXiv 28 Apr 2025.

Builds on

41 of 49 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling29 Aug
  2. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models11 Dec
  3. 2022Large Language Models Can Self-Improve20 Oct
  4. 2022STaR: Bootstrapping Reasoning With Reasoning28 Mar

Other references · 37

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  10. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  11. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  12. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
25 more
  1. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  2. 2023A Survey of Large Language Modelscited by 2,576
  3. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  4. 2024Qwen2 Technical Reportcited by 2,120
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  6. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  8. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  9. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  10. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 427
  11. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  12. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 301
  13. 2023Think before you speak: Training Language Models With Pause Tokenscited by 264
  14. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  15. 2024Reinforcement Learning for LLM Post-Training: A Surveycited by 92
  16. 2020Sample Efficient Reinforcement Learning with REINFORCEcited by 86
  17. 2024Self-playing Adversarial Language Game Enhances LLM Reasoningcited by 83
  18. 2018Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graphcited by 67
  19. 2023Training Chain-of-Thought via Latent-Variable Inferencecited by 47
  20. 2024Putting GPT-4o to the Sword: A Comprehensive Evaluation of Language, Vision, Speech, and Multimodal Proficiencycited by 31
  21. 2017Learning to Learn: Meta-Critic Networks for Sample Efficient Learningcited by 28
  22. 2020MetaCURE: Meta Reinforcement Learning with Empowerment-Driven Explorationcited by 25
  23. 2023Offline Meta Reinforcement Learning with In-Distribution Online Adaptationcited by 12
  24. 2024Hint-before-Solving Prompting: Guiding LLMs to Effectively Utilize Encoded Knowledgecited by 10
  25. 2024Can GPT-O1 Kill All Bugs? An Evaluation of GPT-Family LLMs on QuixBugscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Teaching LLMs to Plan: Logical Chain-of-Thought Instruction Tuning for Symbolic Planningcited by 15
  2. 2025AIPOM: Agent-aware Interactive Planning for Multi-Agent Systemscited by 1
  3. 2026PaperGuide: Making Small Language-Model Paper-Reading Agents More Efficientcited by 0
  4. 2026Unsupervised Post-Training of Foundation Models: A Surveycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Large Language Models Are Self-Taught Reasoners: Enhancing LLM Applications via Tailored Problem-Solving Demonstrations
  2. 2026Mid-Training with Self-Generated Data Improves Reinforcement Learning in Language Models
  3. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing
  4. 2025PLAN-TUNING: Post-Training Language Models to Learn Step-by-Step Planning for Complex Problem Solving
  5. 2025Teaching LLMs to Think Mathematically: A Critical Study of Decision-Making via Optimization
  6. 2025A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning
  7. 2023On the Planning Abilities of Large Language Models : A Critical Investigation
  8. 2026Beyond Meta-Reasoning: Metacognitive Consolidation for Self-Improving LLM Reasoning