mlsanityData to 7 Oct 2026

Accelerating Reinforcement Learning with Learned Skill Priors

arXiv 2010.11944 ↗First posted 22 Oct 2020cs.LG cs.AI cs.ROcited by 220

Not on the map. Why?

Cited by 50 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Intelligent agents rely heavily on prior experience when learning a new task, yet most modern reinforcement learning (RL) approaches learn every task from scratch. One approach for leveraging prior knowledge is to transfer skills learned on prior tasks to the new task. However, as the amount of prior experience increases, the number of transferable skills grows too, making it challenging to explore the full set of available skills during downstream learning. Yet, intuitively, not all skills should be explored with equal probability; for example information about the current state can hint which skills are promising to explore. In this work, we propose to implement this intuition by learning a prior over skills. We propose a deep latent variable model that jointly learns an embedding space of skills and the skill prior from offline agent experience. We then extend common maximum-entropy RL approaches to use skill priors to guide downstream learning. We validate our approach, SPiRL (Skill-Prior RL), on complex navigation and robotic manipulation tasks and show that learned skill priors are essential for effective skill transfer from rich datasets. Videos and code are available at https://clvrai.com/spirl.

Key facts. Cited by 220 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 43; 2023: 46; 2024: 37; 2025: 34; 2026: 34.

220 papers.

Sort:

By area:

  1. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  2. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  3. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  4. 2023Imitating Human Behaviour with Diffusion Modelscited by 260
  5. 2023Goal-Conditioned Imitation Learning using Score-based Diffusion Policiescited by 229
  6. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 211
  7. 2024Behavior Generation with Latent Actionscited by 159
  8. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
  9. 2023FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulationcited by 132
  10. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  11. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learningcited by 105
  12. 2022From Play to Policy: Conditional Behavior Generation from Uncurated Robot Datacited by 103
38 more
  1. 2024HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulationcited by 101
  2. 2025Reinforcement Learning with Action Chunkingcited by 97
  3. 2024Foundation Policies with Hilbert Representationscited by 89
  4. 2023Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidancecited by 85
  5. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learningcited by 81
  6. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learningcited by 77
  7. 2021Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Taskscited by 75
  8. 2023XSkill: Cross Embodiment Skill Discoverycited by 74
  9. 2021Demonstration-Guided Reinforcement Learning with Learned Skillscited by 73
  10. 2023Skill-Mix: a Flexible and Expandable Family of Evaluations for AI modelscited by 64
  11. 2021Accelerating Robotic Reinforcement Learning via Parameterized Action Primitivescited by 63
  12. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 63
  13. 2021Learning to Synthesize Programs as Interpretable and Generalizable Policiescited by 59
  14. 2023Diffusion Reward: Learning Rewards via Conditional Video Diffusioncited by 56
  15. 2022Learning Neuro-Symbolic Skills for Bilevel Planningcited by 55
  16. 2023HYDRA: Hybrid Robot Actions for Imitation Learningcited by 54
  17. 2022Retrieval-Augmented Reinforcement Learningcited by 53
  18. 2022Skill-based Model-based Reinforcement Learningcited by 50
  19. 2023PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Playcited by 48
  20. 2022Dichotomy of Control: Separating What You Can Control from What You Cannotcited by 47
  21. 2022Retrospectives on the Embodied AI Workshopcited by 46
  22. 2021Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularizationcited by 44
  23. 2021Learning Neuro-Symbolic Relational Transition Models for Bilevel Planningcited by 43
  24. 2024RT-Affordance: Affordances are Versatile Intermediate Representations for Robot Manipulationcited by 40
  25. 2021TRAIL: Near-Optimal Imitation Learning with Suboptimal Datacited by 39
  26. 2021Hierarchical Few-Shot Imitation with Skill Transition Modelscited by 37
  27. 2021Hierarchical Skills for Efficient Explorationcited by 37
  28. 2022Skill-based Meta-Reinforcement Learningcited by 36
  29. 2023Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priorscited by 36
  30. 2022Spectral Decomposition Representation for Reinforcement Learningcited by 35
  31. 2023Beyond Reward: Offline Preference-guided Policy Optimizationcited by 35
  32. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
  33. 2021Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedbackcited by 34
  34. 2023Reasoning with Latent Diffusion in Offline Reinforcement Learningcited by 33
  35. 2022LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulationcited by 32
  36. 2023Hyper-Decision Transformer for Efficient Online Policy Adaptationcited by 32
  37. 2024RL-GPT: Integrating Reinforcement Learning and Code-as-policycited by 32
  38. 2022Choreographer: Learning and Adapting Skills in Imaginationcited by 31

50 most cited of 220 All 220 (JSON)