mlsanityData to 14 Oct 2026

Practice Makes Perfect: Planning to Learn Skill Parameter Policies

arXiv 2402.15025 ↗First posted 22 Feb 2024cs.RO cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

One promising approach towards effective robot decision making in complex, long-horizon tasks is to sequence together parameterized skills. We consider a setting where a robot is initially equipped with (1) a library of parameterized skills, (2) an AI planner for sequencing together the skills given a goal, and (3) a very general prior distribution for selecting skill parameters. Once deployed, the robot should rapidly and autonomously learn to improve its performance by specializing its skill parameter selection policy to the particular objects, goals, and constraints in its environment. In this work, we focus on the active learning problem of choosing which skills to practice to maximize expected future task success. We propose that the robot should estimate the competence of each skill, extrapolate the competence (asking: "how much would the competence improve through practice?"), and situate the skill in the task distribution through competence-aware planning. This approach is implemented within a fully autonomous system where the robot repeatedly plans, practices, and learns without any environment resets. Through experiments in simulation, we find that our approach learns effective parameter policies more sample-efficiently than several baselines. Experiments in the real-world demonstrate our approach's ability to handle noise from perception and control and improve the robot's ability to solve two long-horizon mobile-manipulation tasks after a few hours of autonomous practice. Project website: http://ees.csail.mit.edu

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 42 of its 67 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

42 of 67 references.

References · 42

  1. 2023Segment Anythingcited by 8,234
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  3. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  4. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 614
  5. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  6. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
  7. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  8. 2020Integrated Task and Motion Planningcited by 290
  9. 2019Self-Supervised Exploration via Disagreementcited by 265
  10. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
  11. 2020Parrot: Data-Driven Behavioral Priors for Reinforcement Learningcited by 138
  12. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 95
30 more
  1. 2023Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Modelscited by 91
  2. 2018CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learningcited by 79
  3. 2021Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Taskscited by 78
  4. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 73
  5. 2015Deep Reinforcement Learning in Parameterized Action Spacecited by 68
  6. 2023LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discoverycited by 67
  7. 2021Accelerating Robotic Reinforcement Learning via Parameterized Action Primitivescited by 65
  8. 2015Reinforcement Learning with Parameterized Actionscited by 63
  9. 2019Online Replanning in Belief Space for Partially Observable Task and Motion Problemscited by 61
  10. 2013Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robotscited by 58
  11. 2021Learning Symbolic Operators for Task and Motion Planningcited by 58
  12. 2022Learning Neuro-Symbolic Skills for Bilevel Planningcited by 58
  13. 2022Predicate Invention for Bilevel Planningcited by 49
  14. 2020Deep Visual Reasoning: Learning to Predict Action Sequences for Task and Motion Planning from an Initial Scene Imagecited by 48
  15. 2020Learning compositional models of robot skills for task and motion planningcited by 48
  16. 2023Compositional Diffusion-Based Continuous Constraint Solverscited by 40
  17. 2015Simple regret for infinitely many armed banditscited by 39
  18. 2020Reset-Free Lifelong Learning with Skill-Space Planningcited by 38
  19. 2022Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Spacecited by 31
  20. 2024Adaptive Mobile Manipulation for Articulated Objects In the Open Worldcited by 31
  21. 2022Learning Efficient Abstract Planning Models that Choose What to Predictcited by 17
  22. 2023Embodied Lifelong Learning for Task and Motion Planningcited by 17
  23. 2019Anytime Integrated Task and Motion Policies for Stochastic Environmentscited by 13
  24. 2021Active Learning of Abstract Plan Feasibilitycited by 11
  25. 2021Action Priors for Large Action Spaces in Roboticscited by 10
  26. 2022Accelerating Integrated Task and Motion Planning with Neural Feasibility Checkingcited by 10
  27. 2017Active Exploration for Learning Symbolic Representationscited by 8
  28. 2022Efficient Recovery Learning using Model Predictive Meta-Reasoningcited by 8
  29. 2023Embodied Active Learning of Relational State Abstractions for Bilevel Planningcited by 8
  30. 2022Active Task Randomization: Learning Robust Skills via Unsupervised Generation of Diverse and Feasible Taskscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 11; 2026: 4.

23 papers.

Sort:

By area:

  1. 2025A Self-Improving Coding Agentcited by 79
  2. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learningcited by 71
  3. 2024MaestroMotif: Skill Design from Artificial Intelligence Feedbackcited by 24
  4. 2024Trust the PRoC3S: Solving Long-Horizon Robotics Problems with LLMs and Constraint Satisfactioncited by 19
  5. 2024From Pixels to Predicates: Learning Symbolic World Models via Pretrained Vision-Language Modelscited by 11
  6. 2024Adaptive Language-Guided Abstraction from Contrastive Explanationscited by 8
  7. 2024On-Robot Reinforcement Learning with Goal-Contrastive Rewardscited by 8
  8. 2025MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spacescited by 7
  9. 2024Equivariant Action Sampling for Reinforcement Learning and Planningcited by 4
  10. 2025A Systematic Study of Large Language Models for Task and Motion Planning With PDDLStreamcited by 4
  11. 2025SLAP: Shortcut Learning for Abstract Planningcited by 3
  12. 2025Learning Long-Horizon Robot Manipulation Skills via Privileged Actioncited by 2
11 more
  1. 2024Shadow Program Inversion with Differentiable Planning: A Framework for Unified Robot Program Parameter and Trajectory Optimizationcited by 1
  2. 2024Learning to Bridge the Gap: Efficient Novelty Recovery with Planning and Reinforcement Learningcited by 1
  3. 2025Automating Curriculum Learning for Reinforcement Learning using a Skill-Based Bayesian Networkcited by 1
  4. 2025MuTRAP: Multi-trigger Trojans Attacking Robot Task Planning Systemscited by 1
  5. 2025Coloring Between the Lines: Personalization in the Null Space of Planning Constraintscited by 1
  6. 2026Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Modelscited by 1
  7. 2025ActivePusher: Active Learning and Planning with Residual Physics for Nonprehensile Manipulationcited by 0
  8. 2025Empowering LLMs with Parameterized Skills for Adversarial Long-Horizon Planningcited by 0
  9. 2026Neuro-Symbolic Learning for Long-Horizon Task Planning Under Complex Logical Constraintscited by 0
  10. 2026HELP: Human-Efficient Large-Scale Robot Post-Training with Rollout Segmentationcited by 0
  11. 2026Deliberate Practice: Learning Robot Skills under a Budgetcited by 0