mlsanityData to 7 Oct 2026

Demonstration-Guided Reinforcement Learning with Learned Skills

arXiv 2107.10253 ↗First posted 21 Jul 2021cs.LG cs.AI cs.ROcited by 73

Not on the map. Why?

Cited by 21 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Demonstration-guided reinforcement learning (RL) is a promising approach for learning complex behaviors by leveraging both reward feedback and a set of target task demonstrations. Prior approaches for demonstration-guided RL treat every new task as an independent learning problem and attempt to follow the provided demonstrations step-by-step, akin to a human trying to imitate a completely unseen behavior by following the demonstrator's exact muscle movements. Naturally, such learning will be slow, but often new behaviors are not completely unseen: they share subtasks with behaviors we have previously learned. In this work, we aim to exploit this shared subtask structure to increase the efficiency of demonstration-guided RL. We first learn a set of reusable skills from large offline datasets of prior experience collected across many tasks. We then propose Skill-based Learning with Demonstrations (SkiLD), an algorithm for demonstration-guided RL that efficiently leverages the provided demonstrations by following the demonstrated skills instead of the primitive actions, resulting in substantial performance improvements over prior demonstration-guided RL approaches. We validate the effectiveness of our approach on long-horizon maze navigation and complex robot manipulation tasks.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). Builds on 30 of its 49 references that are in the map. First posted on arXiv 21 Jul 2021.

Builds on

30 of 49 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  7. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 763
  8. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  9. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  10. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  11. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  12. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
18 more
  1. 2019Learning Latent Plans from Playcited by 310
  2. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  3. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  4. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 220
  5. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 213
  6. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 203
  7. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
  8. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  9. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  10. 2020Parrot: Data-Driven Behavioral Priors for Reinforcement Learningcited by 136
  11. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124
  12. 2018Neural probabilistic motor primitives for humanoid controlcited by 96
  13. 2020COG: Connecting New Skills to Past Experience with Offline Reinforcement Learningcited by 83
  14. 2017Learning human behaviors from motion capture by adversarial imitationcited by 75
  15. 2020RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learningcited by 57
  16. 2019Dynamics-aware Embeddingscited by 34
  17. 2019Keyframing the Future: Keyframe Discovery for Visual Prediction and Planningcited by 20
  18. 2018CompILE: Compositional Imitation Learning and Executioncited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 15; 2023: 21; 2024: 19; 2025: 9; 2026: 7.

73 papers.

Sort:

By area:

  1. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  2. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 104
  3. 2023Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidancecited by 85
  4. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learningcited by 77
  5. 2022Skill-based Model-based Reinforcement Learningcited by 50
  6. 2024SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deploymentcited by 48
  7. 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 42
  8. 2021TRAIL: Near-Optimal Imitation Learning with Suboptimal Datacited by 39
  9. 2022Skill-based Meta-Reinforcement Learningcited by 36
  10. 2022Spectral Decomposition Representation for Reinforcement Learningcited by 35
  11. 2022Choreographer: Learning and Adapting Skills in Imaginationcited by 31
  12. 2024BUMBLE: Unifying Reasoning and Acting with Vision-Language Models for Building-wide Mobile Manipulationcited by 31
38 more
  1. 2021Learning Transferable Motor Skills with Hierarchical Latent Mixture Policiescited by 26
  2. 2023SPRINT: Scalable Policy Pre-Training via Language Instruction Relabelingcited by 24
  3. 2022Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Roboticscited by 23
  4. 2024Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problemcited by 23
  5. 2023Accelerating Exploration with Unlabeled Prior Datacited by 21
  6. 2023Language-Conditioned Imitation Learning with Base Skill Priors under Unstructured Datacited by 20
  7. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  8. 2024Countering Reward Over-optimization in LLM with Demonstration-Guided Reinforcement Learningcited by 20
  9. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  10. 2022Cross-Domain Transfer via Semantic Skill Imitationcited by 17
  11. 2024Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Modelscited by 15
  12. 2023LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teacherscited by 14
  13. 2023Efficient Planning with Latent Diffusioncited by 11
  14. 2022MO2: Model-Based Offline Optionscited by 10
  15. 2023Learning Robot Manipulation from Cross-Morphology Demonstrationcited by 8
  16. 2023Learning to Discover Skills through Guidancecited by 8
  17. 2024PRIME: Scaffolding Manipulation Tasks with Behavior Primitives for Data-Efficient Imitation Learningcited by 8
  18. 2024Robot Policy Learning with Temporal Optimal Transport Rewardcited by 8
  19. 2023Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deploymentcited by 7
  20. 2024Robust Policy Learning via Offline Skill Diffusioncited by 7
  21. 2025Exploratory Retrieval-Augmented Planning For Continual Embodied Instruction Followingcited by 7
  22. 2022SAFER: Data-Efficient and Safe Reinforcement Learning via Skill Acquisitioncited by 6
  23. 2023Skill-Critic: Refining Learned Skills for Hierarchical Reinforcement Learningcited by 6
  24. 2026Hierarchical Latent Action Modelcited by 6
  25. 2023Continual Robot Learning using Self-Supervised Task Inferencecited by 5
  26. 2023Surgical Gym: A high-performance GPU-based platform for reinforcement learning with surgical robotscited by 5
  27. 2025Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learningcited by 5
  28. 2022CEIP: Combining Explicit and Implicit Priors for Reinforcement Learning with Demonstrationscited by 4
  29. 2023Learning to Solve Tasks with Exploring Prior Behaviourscited by 4
  30. 2023Decoupling Meta-Reinforcement Learning with Gaussian Task Contexts and Skillscited by 4
  31. 2022Priors, Hierarchy, and Information Asymmetry for Skill Transfer in Reinforcement Learningcited by 3
  32. 2022SFP: State-free Priors for Exploration in Off-Policy Reinforcement Learningcited by 3
  33. 2022H-SAUR: Hypothesize, Simulate, Act, Update, and Repeat for Understanding Object Articulations from Interactionscited by 3
  34. 2023Subwords as Skills: Tokenization for Sparse-Reward Reinforcement Learningcited by 3
  35. 2023A Simple Solution for Offline Imitation from Observations and Examples with Possibly Incomplete Trajectoriescited by 3
  36. 2024Opinion-Guided Reinforcement Learningcited by 3
  37. 2024Skills Regularized Task Decomposition for Multi-task Offline Reinforcement Learningcited by 3
  38. 2024SkillTree: Explainable Skill-Based Deep Reinforcement Learning for Long-Horizon Control Taskscited by 3

50 most cited of 73 All 73 (JSON)