mlsanityData to 14 Oct 2026

Meta Learning Shared Hierarchies

arXiv 1710.09767 ↗First posted 26 Oct 2017cs.LGcited by 93

Not on the map. Why?

Abstract (arXiv)

We develop a metalearning approach for learning hierarchically structured policies, improving sample efficiency on unseen tasks through the use of shared primitives---policies that are executed for large numbers of timesteps. Specifically, a set of primitives are shared within a distribution of tasks, and are switched between by task-specific policies. We provide a concrete metric for measuring the strength of such hierarchies, leading to an optimization problem for quickly reaching high reward on unseen tasks. We then present an algorithm to solve this problem end-to-end through the use of any off-the-shelf reinforcement learning method, by repeatedly sampling new tasks and resetting task-specific policies. We successfully discover meaningful motor primitives for the directional movement of four-legged robots, solely by interacting with distributions of mazes. We also demonstrate the transferability of primitives to solve long-timescale sparse-reward obstacle courses, and we enable 3D humanoid robots to robustly walk and crawl with the same policy.

Key facts. Cited by 93 papers in the map (data through 14 Oct 2026). First posted on arXiv 26 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 26; 2022: 28; 2023: 11; 2024: 10; 2025: 11; 2026: 6.

93 papers.

Sort:

By area:

  1. 2021Character Controllers Using Motion VAEscited by 184
  2. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 103
  3. 2022Deep Hierarchical Planning from Pixelscited by 101
  4. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  5. 2025Self-Adapting Language Modelscited by 84
  6. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 73
  7. 2021The Option Keyboard: Combining Skills in Reinforcement Learningcited by 67
  8. 2021Accelerating Robotic Reinforcement Learning via Parameterized Action Primitivescited by 65
  9. 2023Multi-Stage Cable Routing through Hierarchical Imitation Learningcited by 60
  10. 2022Learning Neuro-Symbolic Skills for Bilevel Planningcited by 58
  11. 2022Retrieval-Augmented Reinforcement Learningcited by 53
  12. 2021Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularizationcited by 46
38 more
  1. 2021Discovery of Options via Meta-Learned Subgoalscited by 27
  2. 2022Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discoverycited by 23
  3. 2021How to Train Your MAML to Excel in Few-Shot Classificationcited by 22
  4. 2021Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversitycited by 22
  5. 2022Intelligent problem-solving as integrated hierarchical reinforcement learningcited by 21
  6. 2023Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Modelscited by 21
  7. 2024Hierarchical Multi-agent Reinforcement Learning for Cyber Network Defensecited by 17
  8. 2022On Efficient Reinforcement Learning for Full-length Game of StarCraft IIcited by 14
  9. 2022Federated Learning and Meta Learning: Approaches, Applications, and Directionscited by 12
  10. 2024DrS: Learning Reusable Dense Rewards for Multi-Stage Taskscited by 10
  11. 2025Mobi-π: Mobilizing Your Robot Learning Policycited by 10
  12. 2021Hierarchical Reinforcement Learning for Air Combat at DARPA's AlphaDogfight Trialscited by 8
  13. 2021Context-Specific Representation Abstraction for Deep Option Learningcited by 8
  14. 2022Assistive Teaching of Motor Control Tasks to Humanscited by 8
  15. 2026Temperature as a Meta-Policy: Adaptive Temperature in LLM Reinforcement Learningcited by 8
  16. 2021BR-NS: an Archive-less Approach to Novelty Searchcited by 7
  17. 2022Transfering Hierarchical Structure with Dual Meta Imitation Learningcited by 7
  18. 2022Meta-Learning Parameterized Skillscited by 7
  19. 2022Hierarchical Kickstarting for Skill Transfer in Reinforcement Learningcited by 7
  20. 2022ASPiRe:Adaptive Skill Priors for Reinforcement Learningcited by 7
  21. 2024Reinforcement learning with learned gadgets to tackle hard quantum problems on real hardwarecited by 7
  22. 2021Toward Robust Long Range Policy Transfercited by 6
  23. 2021Training Transition Policies via Distribution Matching for Complex Taskscited by 6
  24. 2022SkillS: Adaptive Skill Sequencing for Efficient Temporally-Extended Explorationcited by 6
  25. 2023Successor-Predecessor Intrinsic Explorationcited by 6
  26. 2021Discovering Generalizable Skills via Automated Generation of Diverse Taskscited by 5
  27. 2021Omni-Training: Bridging Pre-Training and Meta-Training for Few-Shot Learningcited by 5
  28. 2021Provable Hierarchy-Based Meta-Reinforcement Learningcited by 5
  29. 2025Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learningcited by 5
  30. 2022Developing cooperative policies for multi-stage reinforcement learning taskscited by 4
  31. 2022Challenges to Solving Combinatorially Hard Long-Horizon Deep RL Taskscited by 4
  32. 2022Skill-Based Reinforcement Learning with Intrinsic Reward Matchingcited by 4
  33. 2022Assessing Quality-Diversity Neuro-Evolution Algorithms Performance in Hard Exploration Problemscited by 4
  34. 2021Hierarchical Primitive Composition: Simultaneous Activation of Skills with Inconsistent Action Dimensions in Multiple Hierarchiescited by 3
  35. 2025Task-Aware Virtual Training: Enhancing Generalization in Meta-Reinforcement Learning for Out-of-Distribution Taskscited by 3
  36. 2021Learning Setup Policies: Reliable Transition Between Locomotion Behaviourscited by 2
  37. 2021Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithmcited by 2
  38. 2021Adaptive Approach Phase Guidance for a Hypersonic Glider via Reinforcement Meta Learningcited by 2

50 most cited of 93 All 93 (JSON)