mlsanityData to 7 Oct 2026

Skill-based Model-based Reinforcement Learning

arXiv 2207.07560 ↗First posted 15 Jul 2022cs.LG cs.AI cs.ROcited by 50

Not on the map. Why?

Abstract (arXiv)

Model-based reinforcement learning (RL) is a sample-efficient way of learning complex behaviors by leveraging a learned single-step dynamics model to plan actions in imagination. However, planning every action for long-horizon tasks is not practical, akin to a human planning out every muscle movement. Instead, humans efficiently plan with high-level skills to solve complex tasks. From this intuition, we propose a Skill-based Model-based RL framework (SkiMo) that enables planning in the skill space using a skill dynamics model, which directly predicts the skill outcomes, rather than predicting all small details in the intermediate states, step by step. For accurate and efficient long-term planning, we jointly learn the skill dynamics model and a skill repertoire from prior experience. We then harness the learned skill dynamics model to accurately simulate and plan over long horizons in the skill space, which enables efficient downstream learning of long-horizon, sparse reward tasks. Experimental results in navigation and manipulation domains show that SkiMo extends the temporal horizon of model-based approaches and improves the sample efficiency for both model-based RL and skill-based RL. Code and videos are available at https://clvrai.com/skimo

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 38 of its 56 references that are in the map. First posted on arXiv 15 Jul 2022.

Builds on

38 of 56 references.

References · 38

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  4. 2018World Modelscited by 1,444
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2015Prioritized Experience Replaycited by 1,070
  7. 2018DeepMind Control Suitecited by 935
  8. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  9. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  10. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  11. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  12. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
26 more
  1. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  2. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  3. 2019Learning Latent Plans from Playcited by 310
  4. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  5. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 268
  6. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 220
  7. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
  8. 2021Discovering and Achieving Goals via World Modelscited by 131
  9. 2020Model-Based Offline Planningcited by 126
  10. 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 122
  11. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrationscited by 114
  12. 2018Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Controlcited by 104
  13. 2019Exploring Model-based Planning with Policy Networkscited by 90
  14. 2019IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Datacited by 85
  15. 2021Demonstration-Guided Reinforcement Learning with Learned Skillscited by 73
  16. 2019IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Taskscited by 68
  17. 2019Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?cited by 66
  18. 2021Accelerating Robotic Reinforcement Learning via Parameterized Action Primitivescited by 63
  19. 2018TACO: Learning Task Decomposition via Temporal Alignment for Controlcited by 61
  20. 2020Learning Robot Skills with Temporal Variational Inferencecited by 54
  21. 2021Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularizationcited by 44
  22. 2020Reset-Free Lifelong Learning with Skill-Space Planningcited by 38
  23. 2021Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoningcited by 25
  24. 2020Latent Skill Planning for Exploration and Transfercited by 19
  25. 2021Learning Task Decomposition with Ordered Memory Policy Networkcited by 15
  26. 2021Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Taskscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 13; 2024: 13; 2025: 10; 2026: 12.

50 papers.

Sort:

By area:

  1. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
  2. 2023Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidancecited by 85
  3. 2023Diffusion Models for Reinforcement Learning: A Surveycited by 84
  4. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learningcited by 81
  5. 2024Simple Hierarchical Planning with Diffusioncited by 71
  6. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbonecited by 71
  7. 2023KITE: Keypoint-Conditioned Policies for Semantic Manipulationcited by 39
  8. 2023Model-Based Runtime Monitoring with Interactive Imitation Learningcited by 37
  9. 2024Hierarchical World Models as Visual Whole-Body Humanoid Controllerscited by 24
  10. 2023DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physicscited by 23
  11. 2023Language-Conditioned Imitation Learning with Base Skill Priors under Unstructured Datacited by 20
  12. 2023Contrastive Language, Action, and State Pre-training for Robot Learningcited by 15
38 more
  1. 2022Hyperbolic VAE via Latent Gaussian Distributionscited by 14
  2. 2024EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Datacited by 13
  3. 2025Prior-Guided Diffusion Planning for Offline Reinforcement Learningcited by 13
  4. 2025Scalable Offline Model-Based RL with Action Chunkscited by 12
  5. 2025Learning Massively Multitask World Models for Continuous Controlcited by 11
  6. 2023Learning to Discover Skills through Guidancecited by 8
  7. 2023DreamSmooth: Improving Model-based Reinforcement Learning via Reward Smoothingcited by 8
  8. 2024A call for embodied AIcited by 7
  9. 2026Lifting Traces to Logic: Programmatic Skill Induction with Neuro-Symbolic Learning for Long-Horizon Agentic Taskscited by 7
  10. 2023Learning Sequential Acquisition Policies for Robot-Assisted Feedingcited by 6
  11. 2023DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physicscited by 6
  12. 2024Generalize by Touching: Tactile Ensemble Skill Transfer for Robotic Furniture Assemblycited by 6
  13. 2025Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learningcited by 5
  14. 2025Temporal Distance-aware Transition Augmentation for Offline Model-based Reinforcement Learningcited by 5
  15. 2026Chain-of-Goals Hierarchical Policy for Long-Horizon Offline Goal-Conditioned RLcited by 5
  16. 2023Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learningcited by 4
  17. 2024Learning Abstract World Model for Value-preserving Planning with Optionscited by 4
  18. 2026Compositional Planning with Jumpy World Modelscited by 4
  19. 2025Dynamic Contrastive Skill Learning with State-Transition Based Skill Clustering and Dynamic Length Adjustmentcited by 3
  20. 2025Modeling Unseen Environments with Language-guided Composable Causal Components in Reinforcement Learningcited by 3
  21. 2026IMWM: Intuition Models Complement World Models for Latent Planningcited by 3
  22. 2024I Know How: Combining Prior Policies to Solve New Taskscited by 2
  23. 2024Goal-Reaching Policy Learning from Non-Expert Observations via Effective Subgoal Guidancecited by 2
  24. 2025Learning Skills from Action-Free Videoscited by 2
  25. 2024SOAP-RL: Sequential Option Advantage Propagation for Reinforcement Learning in POMDP Environmentscited by 1
  26. 2024Skill-Enhanced Reinforcement Learning Acceleration from Heterogeneous Demonstrationscited by 1
  27. 2026LAR-MoE: Latent-Aligned Routing for Mixture of Experts in Robotic Imitation Learningcited by 1
  28. 2026Zero-shot Imitation Learning by Latent Topology Mappingcited by 1
  29. 2026Recursive Value Learning for Long-Horizon Offline Goal-Conditioned RLcited by 1
  30. 2024Skill-aware Mutual Information Optimisation for Generalisation in Reinforcement Learningcited by 0
  31. 2024Offline Policy Learning via Skill-step Abstraction for Long-horizon Goal-Conditioned Taskscited by 0
  32. 2025AMPED: Adaptive Multi-objective Projection for balancing Exploration and skill Diversificationcited by 0
  33. 2025FAST: Similarity-based Knowledge Transfer for Efficient Policy Learningcited by 0
  34. 2026Governed Capability Evolution: Lifecycle-Time Compatibility Checking and Rollback for AI-Component-Based Systems, with a Proof-of-Concept Evaluation on Embodied Agentscited by 0
  35. 2026Hierarchical Behaviour Spacescited by 0
  36. 2026Regression Test Selection for Updated Capability Modules in Compositional ML Systems via Atomic-Quality Probescited by 0
  37. 2026Switching Successor Measures for Hierarchical Zero-shot Reinforcement Learningcited by 0
  38. 2026From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Interventioncited by 0