mlsanityData to 7 Oct 2026

Provable Hierarchical Imitation Learning via EM

arXiv 2010.03133 ↗First posted 7 Oct 2020cs.LG stat.MLcited by 10

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Due to recent empirical successes, the options framework for hierarchical reinforcement learning is gaining increasing popularity. Rather than learning from rewards which suffers from the curse of dimensionality, we consider learning an options-type hierarchical policy from expert demonstrations. Such a problem is referred to as hierarchical imitation learning. Converting this problem to parameter inference in a latent variable model, we theoretically characterize the EM approach proposed by Daniel et al. (2016). The population level algorithm is analyzed as an intermediate step, which is nontrivial due to the samples being correlated. If the expert policy can be parameterized by a variant of the options framework, then under regularity conditions, we prove that the proposed algorithm converges with high probability to a norm ball around the true parameter. To our knowledge, this is the first performance guarantee for an hierarchical imitation learning algorithm that only observes primitive state-action pairs.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 18 of its 50 references that are in the map. First posted on arXiv 7 Oct 2020.

Builds on

18 of 50 references.

References · 18

  1. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  2. 2016The Option-Critic Architecturecited by 433
  3. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  4. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  5. 2017Non-convex Optimization for Machine Learningcited by 132
  6. 2014Statistical guarantees for the EM algorithm: From population to sample-based analysiscited by 131
  7. 2016A Deep Hierarchical Approach to Lifelong Learning in Minecraftcited by 90
  8. 2018Hierarchical Imitation and Reinforcement Learningcited by 88
  9. 2018TACO: Learning Task Decomposition via Temporal Alignment for Controlcited by 61
  10. 2017Multi-Level Discovery of Deep Optionscited by 50
  11. 2017DDCO: Discovery of Deep Continuous Options for Robot Learning from Demonstrationscited by 43
  12. 2016Ten Steps of EM Suffice for Mixtures of Two Gaussianscited by 40
6 more
  1. 2018Singularity, Misspecification, and the Convergence Rate of EMcited by 38
  2. 2016Global analysis of Expectation Maximization for mixtures of two Gaussianscited by 31
  3. 2015Regularized EM Algorithms: A Unified Framework and Statistical Guaranteescited by 19
  4. 2017Exploration--Exploitation in MDPs with Optionscited by 19
  5. 2015Statistical and Computational Guarantees for the Baum-Welch Algorithmcited by 11
  6. 2013On learning parametric-output HMMscited by 5

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 1; 2023: 2; 2024: 4; 2025: 1.

10 papers.

  1. 2021Adversarial Option-Aware Hierarchical Imitation Learningcited by 19
  2. 2022Option-Aware Adversarial Inverse Reinforcement Learning for Robotic Controlcited by 16
  3. 2023Hierarchical Imitation Learning with Vector Quantized Modelscited by 15
  4. 2024IDIL: Imitation Learning of Intent-Driven Expert Behaviorcited by 5
  5. 2021Online Baum-Welch algorithm for Hierarchical Imitation Learningcited by 3
  6. 2025Hierarchical Imitation Learning of Team Behavior from Heterogeneous Demonstrationscited by 2
  7. 2024SOAP-RL: Sequential Option Advantage Propagation for Reinforcement Learning in POMDP Environmentscited by 1
  8. 2023CasIL: Cognizing and Imitating Skills via a Dual Cognition-Action Architecturecited by 0
  9. 2024Offline Imitation of Badminton Player Behavior via Experiential Contexts and Brownian Motioncited by 0
  10. 2024Interpretable Robotic Manipulation from Languagecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2019Hierarchical Variational Imitation Learning of Control Programs
  2. 2018Accelerating Imitation Learning with Predictive Models
  3. 2022Deconfounding Imitation Learning with Variational Inference
  4. 2018Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Information
  5. 2020HILONet: Hierarchical Imitation Learning from Non-Aligned Observations
  6. 2020Active Hierarchical Imitation and Reinforcement Learning
  7. 2021Hyperparameter Selection for Imitation Learning
  8. 2026Infeasible optimization problems and the hierarchical augmented Lagrangian method in imitation learning