mlsanityData to 7 Oct 2026

Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Information

arXiv 1810.01266 ↗First posted 29 Sep 2018cs.LG cs.AI stat.MLcited by 41

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

The use of imitation learning to learn a single policy for a complex task that has multiple modes or hierarchical structure can be challenging. In fact, previous work has shown that when the modes are known, learning separate policies for each mode or sub-task can greatly improve the performance of imitation learning. In this work, we discover the interaction between sub-tasks from their resulting state-action trajectory sequences using a directed graphical model. We propose a new algorithm based on the generative adversarial imitation learning framework which automatically learns sub-task policies from unsegmented demonstrations. Our approach maximizes the directed information flow in the graphical model between sub-task latent variables and their generated trajectories. We also show how our approach connects with the existing Options framework, which is commonly used to learn hierarchical policies.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 29 Sep 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 2; 2020: 2; 2021: 8; 2022: 9; 2023: 9; 2024: 5; 2025: 5; 2026: 1.

41 papers.

Sort:

By area:

  1. 2023Motion Planning for Autonomous Driving: The State of the Art and Future Perspectivescited by 112
  2. 2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 57
  3. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variablescited by 50
  4. 2021MORAL: Aligning AI with Human Norms through Multi-Objective Reinforced Active Learningcited by 24
  5. 2022Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Roboticscited by 23
  6. 2022Learning Options via Compressioncited by 22
  7. 2021Adversarial Option-Aware Hierarchical Imitation Learningcited by 19
  8. 2021Discovering Diverse Solutions in Deep Reinforcement Learning by Maximizing State-Action-Based Mutual Informationcited by 18
  9. 2022Option-Aware Adversarial Inverse Reinforcement Learning for Robotic Controlcited by 16
  10. 2023Hierarchical Imitation Learning with Vector Quantized Modelscited by 15
  11. 2023Multi-task Hierarchical Adversarial Inverse Reinforcement Learningcited by 15
  12. 2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
29 more
  1. 2021Motion Planning by Learning the Solution Manifold in Trajectory Optimizationcited by 13
  2. 2021Inverse Contextual Bandits: Learning How Behavior Evolves over Timecited by 12
  3. 2024Language-guided Skill Learning with Temporal Variational Inferencecited by 11
  4. 2022Learning from Guided Play: Improving Exploration for Adversarial Imitation Learning with Simple Auxiliary Taskscited by 10
  5. 2025From Simple to Complex Skills: The Case of In-Hand Object Reorientationcited by 8
  6. 2020Domain-Adversarial and Conditional State Space Model for Imitation Learningcited by 7
  7. 2022Transfering Hierarchical Structure with Dual Meta Imitation Learningcited by 7
  8. 2023Automated Task-Time Interventions to Improve Teamwork using Imitation Learningcited by 7
  9. 2023GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimationcited by 6
  10. 2022Bayesian Nonparametrics for Offline Skill Discoverycited by 5
  11. 2022SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environmentscited by 5
  12. 2022Learning Multi-Task Transferable Rewards via Variational Inverse Reinforcement Learningcited by 5
  13. 2024IDIL: Imitation Learning of Intent-Driven Expert Behaviorcited by 5
  14. 2023Deep Probabilistic Movement Primitives with a Bayesian Aggregatorcited by 4
  15. 2024Identifying Selections for Unsupervised Subtask Discoverycited by 4
  16. 2025Stabilizing Long-term Multi-turn Reinforcement Learning with Gated Rewardscited by 4
  17. 2020oIRL: Robust Adversarial Inverse Reinforcement Learning with Temporally Extended Actionscited by 3
  18. 2021Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learningcited by 3
  19. 2023Explaining by Imitating: Understanding Decisions by Interpretable Policy Learningcited by 3
  20. 2024BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human Racing Gameplaycited by 3
  21. 2025Hierarchical Imitation Learning of Team Behavior from Heterogeneous Demonstrationscited by 2
  22. 2023Diverse Policy Optimization for Structured Action Spacecited by 1
  23. 2023Boosting Reinforcement Learning and Planning with Demonstrations: A Surveycited by 1
  24. 2025Learning Task-Agnostic Motifs to Capture the Continuous Nature of Animal Behaviorcited by 1
  25. 2025Learning Parameterized Skills from Demonstrationscited by 1
  26. 2019Mature GAIL: Imitation Learning for Low-level and High-dimensional Input using Global Encoder and Cost Transformationcited by 0
  27. 2022Data-driven prediction of Air Traffic Controllers reactions to resolving conflictscited by 0
  28. 2024Interpretable Robotic Manipulation from Languagecited by 0
  29. 2026Learning Multi-Timescale Abstractions for Hierarchical Combinatorial Planningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2019Adversarial Imitation Learning from Incomplete Demonstrations
  2. 2022Plan Your Target and Learn Your Skills: Transferable State-Only Imitation Learning via Decoupled Policy Optimization
  3. 2022Latent Policies for Adversarial Imitation Learning
  4. 2024Efficient Imitation Learning with Conservative World Models
  5. 2019Hierarchical Variational Imitation Learning of Control Programs
  6. 2023Exploring Gradient Explosion in Generative Adversarial Imitation Learning: A Probabilistic Perspective
  7. 2020f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning
  8. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations