mlsanityData to 14 Oct 2026

Transfering Hierarchical Structure with Dual Meta Imitation Learning

arXiv 2201.11981 ↗First posted 28 Jan 2022cs.RO cs.AIcited by 7

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Hierarchical Imitation Learning (HIL) is an effective way for robots to learn sub-skills from long-horizon unsegmented demonstrations. However, the learned hierarchical structure lacks the mechanism to transfer across multi-tasks or to new tasks, which makes them have to learn from scratch when facing a new situation. Transferring and reorganizing modular sub-skills require fast adaptation ability of the whole hierarchical structure. In this work, we propose Dual Meta Imitation Learning (DMIL), a hierarchical meta imitation learning method where the high-level network and sub-skills are iteratively meta-learned with model-agnostic meta-learning. DMIL uses the likelihood of state-action pairs from each sub-skill as the supervision for the high-level network adaptation, and use the adapted high-level network to determine different data set for each sub-skill adaptation. We theoretically prove the convergence of the iterative training process of DMIL and establish the connection between DMIL and Expectation-Maximization algorithm. Empirically, we achieve state-of-the-art few-shot imitation learning performance on the Meta-world citemetaworld benchmark and competitive results on long-horizon tasks of Kitchen environments.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 39 of its 57 references that are in the map. First posted on arXiv 28 Jan 2022.

Builds on

39 of 57 references.

Closest ancestors · 3

Same direction, newest first

  1. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning25 Oct
  2. 2018One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks25 Oct
  3. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skills26 Feb

Other references · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2016Generative Adversarial Imitation Learningcited by 1,597
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  6. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  7. 2019When Does Label Smoothing Help?cited by 988
  8. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 860
  9. 2018An Algorithmic Perspective on Imitation Learningcited by 459
  10. 2019Learning Latent Plans from Playcited by 316
  11. 2018Probabilistic Model-Agnostic Meta-Learningcited by 256
  12. 2017One-Shot Imitation Learningcited by 233
24 more
  1. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 228
  2. 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
  3. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 148
  4. 2019On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithmscited by 132
  5. 2017Stochastic Neural Networks for Hierarchical Reinforcement Learningcited by 124
  6. 2019MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policiescited by 108
  7. 2019Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulationcited by 103
  8. 2017Meta Learning Shared Hierarchiescited by 93
  9. 2018Hierarchical Imitation and Reinforcement Learningcited by 88
  10. 2017Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learningcited by 77
  11. 2015Deep Reinforcement Learning in Parameterized Action Spacecited by 68
  12. 2019Hierarchically Structured Meta-learningcited by 68
  13. 2018Latent Space Policies for Hierarchical Reinforcement Learningcited by 67
  14. 2018Modular meta-learningcited by 51
  15. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variablescited by 50
  16. 2017DDCO: Discovery of Deep Continuous Options for Robot Learning from Demonstrationscited by 43
  17. 2018Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Informationcited by 41
  18. 2017OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learningcited by 40
  19. 2021Hierarchical Few-Shot Imitation with Skill Transition Modelscited by 37
  20. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learningcited by 36
  21. 2018Hierarchical visuomotor control of humanoidscited by 30
  22. 2015Early Stopping is Nonparametric Variational Inferencecited by 23
  23. 2021Adversarial Option-Aware Hierarchical Imitation Learningcited by 19
  24. 2020Gradient-EM Bayesian Meta-learningcited by 7

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 3.

7 papers.

  1. 2025VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Modelscited by 36
  2. 2023Multi-task Hierarchical Adversarial Inverse Reinforcement Learningcited by 15
  3. 2025Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuningcited by 8
  4. 2023Skill-Critic: Refining Learned Skills for Hierarchical Reinforcement Learningcited by 6
  5. 2024Imitation Learning from Suboptimal Demonstrations via Meta-Learning An Action Rankercited by 1
  6. 2024On the benefits of pixel-based hierarchical policies for task generalizationcited by 0
  7. 2025Robot Policy Transfer with Online Demonstrations: An Active Reinforcement Learning Approachcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation
  2. 2020HILONet: Hierarchical Imitation Learning from Non-Aligned Observations
  3. 2024One-Shot Robust Imitation Learning for Long-Horizon Visuomotor Tasks from Unsegmented Demonstrations
  4. 2026Dual-Process Atomic Skill Learning: Decoupling Semantic Reasoning and Real-Time Control
  5. 2020Scalable Multi-Task Imitation Learning with Autonomous Improvement
  6. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learning
  7. 2026Dynamics-Aware Meta-Imitation for Generalization to Unseen Robotic Manipulation
  8. 2022An Independently Learnable Hierarchical Model for Bilateral Control-Based Imitation Learning Applications