mlsanityData to 7 Oct 2026

Multi-task Hierarchical Adversarial Inverse Reinforcement Learning

arXiv 2305.12633 ↗First posted 22 May 2023cs.LGcited by 15

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Multi-task Imitation Learning (MIL) aims to train a policy capable of performing a distribution of tasks based on multi-task expert demonstrations, which is essential for general-purpose robots. Existing MIL algorithms suffer from low data efficiency and poor performance on complex long-horizontal tasks. We develop Multi-task Hierarchical Adversarial Inverse Reinforcement Learning (MH-AIRL) to learn hierarchically-structured multi-task policies, which is more beneficial for compositional tasks with long horizons and has higher expert data efficiency through identifying and transferring reusable basic skills across tasks. To realize this, MH-AIRL effectively synthesizes context-based multi-task learning, AIRL (an IL approach), and hierarchical policy learning. Further, MH-AIRL can be adopted to demonstrations without the task or skill annotations (i.e., state-action pairs only) which are more accessible in practice. Theoretical justifications are provided for each module of MH-AIRL, and evaluations on challenging multi-task settings demonstrate superior performance and transferability of the multi-task policies learned with MH-AIRL as compared to SOTA MIL baselines.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 27 of its 63 references that are in the map. First posted on arXiv 22 May 2023.

Builds on

27 of 63 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Option-Aware Adversarial Inverse Reinforcement Learning for Robotic Control5 Oct
  2. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variables20 Sep

Other references · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  7. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  8. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  9. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  10. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  11. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  12. 2017Stochastic Neural Networks for Hierarchical Reinforcement Learningcited by 124
13 more
  1. 2019Stacked Capsule Autoencoderscited by 82
  2. 2019DeepPool: Distributed Model-free Algorithm for Ride-sharing using Deep Reinforcement Learningcited by 54
  3. 2019DAC: The Double Actor-Critic Architecture for Learning Optionscited by 43
  4. 2018Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Informationcited by 41
  5. 2018One-Shot Hierarchical Imitation Learning of Compound Visuomotor Taskscited by 32
  6. 2020Scalable Multi-Task Imitation Learning with Autonomous Improvementcited by 21
  7. 2021Adversarial Option-Aware Hierarchical Imitation Learningcited by 19
  8. 2021Communication Efficient Parallel Reinforcement Learningcited by 10
  9. 2019LSTM vs. GRU vs. Bidirectional RNN for script generationcited by 7
  10. 2022Transfering Hierarchical Structure with Dual Meta Imitation Learningcited by 7
  11. 2021Intrinsically Motivated Open-Ended Multi-Task Learning Using Transfer Learning to Discover Task Hierarchycited by 4
  12. 2022SKILL-IL: Disentangling Skill and Knowledge in Multitask Imitation Learningcited by 4
  13. 2022Classical Simulation of Variational Quantum Classifiers using Tensor Ringscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 6; 2025: 2; 2026: 5.

15 papers.

Sort:

By area:

  1. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  2. 2024Computational Teaching for Driving via Multi-Task Imitation Learningcited by 6
  3. 2023Hierarchical Deep Counterfactual Regret Minimizationcited by 4
  4. 2024Diversifying Policy Behaviors with Extrinsic Behavioral Curiositycited by 4
  5. 2024Identifying Selections for Unsupervised Subtask Discoverycited by 4
  6. 2023Cloud-Based Hierarchical Imitation Learning for Scalable Transfer of Construction Skills from Human Workers to Assisting Robotscited by 2
  7. 2025Consistent Zero-Shot Imitation with Contrastive Goal Inferencecited by 1
  8. 2026Offline Discovery of Interpretable Skills from Multi-Task Trajectoriescited by 1
  9. 2026Compositional Transduction with Latent Analogies for Offline Goal-Conditioned Reinforcement Learningcited by 1
  10. 2024Multi Task Inverse Reinforcement Learning for Common Sense Rewardcited by 0
  11. 2024On the benefits of pixel-based hierarchical policies for task generalizationcited by 0
  12. 2025RoboTidy : A 3D Gaussian Splatting Household Tidying Benchmark for Embodied Navigation and Actioncited by 0
3 more
  1. 2026ConTraIRL: Factorized Contrastive Abstractions for Transferable IRLcited by 0
  2. 2026Generalize and Guide: Decomposing Rewards for Few-Shot Inverse Reinforcement Learningcited by 0
  3. 2026Scalable Multi-Task Inverse Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Meta-Adversarial Inverse Reinforcement Learning for Decision-making Tasks
  2. 2020Off-Policy Adversarial Inverse Reinforcement Learning
  3. 2020Scalable Multi-Agent Inverse Reinforcement Learning via Actor-Attention-Critic
  4. 2019Multi-Agent Adversarial Inverse Reinforcement Learning
  5. 2024Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignment
  6. 2018Multi-task Maximum Entropy Inverse Reinforcement Learning
  7. 2024Rethinking Adversarial Inverse Reinforcement Learning: Policy Imitation, Transferable Reward Recovery and Algebraic Equilibrium Proof
  8. 2019Decision Making for Autonomous Driving via Augmented Adversarial Inverse Reinforcement Learning