mlsanityData to 7 Oct 2026

Modular Adaptive Policy Selection for Multi-Task Imitation Learning through Task Division

arXiv 2203.14855 ↗First posted 28 Mar 2022cs.LG cs.ROcited by 2

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Deep imitation learning requires many expert demonstrations, which can be hard to obtain, especially when many tasks are involved. However, different tasks often share similarities, so learning them jointly can greatly benefit them and alleviate the need for many demonstrations. But, joint multi-task learning often suffers from negative transfer, sharing information that should be task-specific. In this work, we introduce a method to perform multi-task imitation while allowing for task-specific features. This is done by using proto-policies as modules to divide the tasks into simple sub-behaviours that can be shared. The proto-policies operate in parallel and are adaptively chosen by a selector mechanism that is jointly trained with the modules. Experiments on different sets of tasks show that our method improves upon the accuracy of single agents, task-conditioned and multi-headed multi-task agents, as well as state-of-the-art meta learning agents. We also demonstrate its ability to autonomously divide the tasks into both shared and task-specific sub-behaviours.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 40 of its 54 references that are in the map. First posted on arXiv 28 Mar 2022.

Builds on

40 of 54 references.

Closest ancestors · 2

Same direction, newest first

  1. 2020Scalable Multi-Task Imitation Learning with Autonomous Improvement25 Feb
  2. 2018Shared Multi-Task Imitation Learning for Indoor Self-Navigation14 Aug

Other references · 38

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2016OpenAI Gymcited by 1,684
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  8. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  9. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  10. 2017A Survey on Multi-Task Learningcited by 838
  11. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  12. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
26 more
  1. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  2. 2015Understanding Neural Networks Through Deep Visualizationcited by 332
  3. 2018How to train your MAMLcited by 306
  4. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  5. 2017One-Shot Imitation Learningcited by 232
  6. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  7. 2019AdaShare: Learning What To Share For Efficient Deep Multi-Task Learningcited by 159
  8. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  9. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  10. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 138
  11. 2017Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learningcited by 125
  12. 2019Task-Driven Modular Networks for Zero-Shot Compositional Learningcited by 121
  13. 2018Driving Policy Transfer via Modularity and Abstractioncited by 91
  14. 2018ProMP: Proximal Meta-Policy Searchcited by 91
  15. 2017Robust Imitation of Diverse Behaviorscited by 85
  16. 2019Branched Multi-Task Networks: Deciding What Layers To Sharecited by 76
  17. 2017Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Netscited by 64
  18. 2018Modular Networks: Learning to Decompose Neural Computationcited by 62
  19. 2018Modular meta-learningcited by 51
  20. 2018A survey on policy search algorithms for learning robot controllers in a handful of trialscited by 33
  21. 2018Task-Oriented Hand Motion Retargeting for Dexterous Manipulation Imitationcited by 25
  22. 2020Meta Reinforcement Learning with Autonomous Inference of Subtask Dependenciescited by 25
  23. 2017Benchmark Environments for Multitask Learning in Continuous Domainscited by 13
  24. 2021Adversarial Imitation Learning with Trajectorial Augmentation and Correctioncited by 11
  25. 2021Modularity in Reinforcement Learning via Algorithmic Independence in Credit Assignmentcited by 6
  26. 2020OCEAN: Online Task Inference for Compositional Tasks with Context Adaptationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 0; 2026: 1.

2 papers.

  1. 2023RObotic MAnipulation Network (ROMAN) -- Hybrid Hierarchical Learning for Solving Complex Sequential Taskscited by 0
  2. 2026Dynamic Full-body Motion Agent with Object Interaction via Blending Pre-trained Modular Controllerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning
  2. 2024Active Fine-Tuning of Multi-Task Policies
  3. 2021Towards More Generalizable One-shot Visual Imitation Learning
  4. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learning
  5. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances
  6. 2023Multi-task Hierarchical Adversarial Inverse Reinforcement Learning
  7. 2022Transfering Hierarchical Structure with Dual Meta Imitation Learning
  8. 2021Intrinsically Motivated Open-Ended Multi-Task Learning Using Transfer Learning to Discover Task Hierarchy