mlsanityData to 13 Oct 2026

Discriminator-Guided Model-Based Offline Imitation Learning

arXiv 2207.00244 ↗First posted 1 Jul 2022cs.LG cs.AIcited by 17

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Offline imitation learning (IL) is a powerful method to solve decision-making problems from expert demonstrations without reward labels. Existing offline IL methods suffer from severe performance degeneration under limited expert data. Including a learned dynamics model can potentially improve the state-action space coverage of expert data, however, it also faces challenging issues like model approximation/generalization errors and suboptimality of rollout data. In this paper, we propose the Discriminator-guided Model-based offline Imitation Learning (DMIL) framework, which introduces a discriminator to simultaneously distinguish the dynamics correctness and suboptimality of model rollout data against real expert demonstrations. DMIL adopts a novel cooperative-yet-adversarial learning strategy, which uses the discriminator to guide and couple the learning process of the policy and dynamics model, resulting in improved model performance and robustness. Our framework can also be extended to the case when demonstrations contain a large proportion of suboptimal data. Experimental results show that DMIL and its extension achieve superior performance and robustness compared to state-of-the-art offline IL methods under small datasets.

Key facts. Cited by 17 papers in the map (data through 13 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 39 of its 61 references that are in the map. First posted on arXiv 1 Jul 2022.

Builds on

39 of 61 references.

Closest ancestors · 22

Same direction, newest first

  1. 2021On Covariate Shift of Latent Confounders in Imitation and Reinforcement Learning13 Oct
  2. 2021Implicit Behavioral Cloning1 Sep
  3. 2021Visual Adversarial Imitation Learning using Variational Models16 Jul
  4. 2021IQ-Learn: Inverse soft-Q Learning for Imitation23 Jun
  5. 2021Mitigating Covariate Shift in Imitation Learning via Offline Data Without Great Coverage6 Jun
  6. 2021No Need for Interactions: Robust Model-Based Imitation Learning using Neural ODE3 Apr
  7. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap4 Mar
  8. 2021MobILE: Model-Based Imitation Learning From Observation Alone22 Feb
  9. 2021Feedback in Imitation Learning: The Three Regimes of Covariate Shift4 Feb
  10. 2021Robust Maximum Entropy Behavior Cloning4 Jan
  11. 2020Offline Learning from Demonstrations and Unlabeled Experience27 Nov
  12. 2020SAFARI: Safe and Active Robot Imitation Learning with Imagination18 Nov
10 more
  1. 2020Toward the Fundamental Limits of Imitation Learning13 Sep
  2. 2020Strictly Batch Imitation Learning by Energy-based Distribution Matching25 Jun
  3. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
  4. 2019SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards27 May
  5. 2018HG-DAgger: Interactive Imitation Learning with Human Experts5 Oct
  6. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow1 Oct
  7. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  8. 2018Accelerating Imitation Learning with Predictive Models12 Jun
  9. 2018Imitating Latent Policies from Observation21 May
  10. 2018Behavioral Cloning from Observation4 May

Other references · 17

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,336
  2. 2016Generative Adversarial Imitation Learningcited by 1,577
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,415
  4. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  5. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 756
  6. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 754
  7. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 704
  8. 2020MOPO: Model-based Offline Policy Optimizationcited by 632
  9. 2019On Evaluating Adversarial Robustnesscited by 447
  10. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 366
  11. 2019DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Correctionscited by 254
  12. 2021Constraints Penalized Q-learning for Safe Offline Reinforcement Learningcited by 90
5 more
  1. 2021DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learningcited by 47
  2. 2021Model-Based Offline Planning with Trajectory Pruningcited by 30
  3. 2021Offline Reinforcement Learning with Soft Behavior Regularizationcited by 28
  4. 2020Semi-supervised reward learning for offline reinforcement learningcited by 22
  5. 2017A Fast Integrated Planning and Control Framework for Autonomous Driving via Imitation Learningcited by 20

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 4; 2024: 7; 2025: 5; 2026: 1.

17 papers.

Sort:

By area:

  1. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  2. 2023Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learningcited by 27
  3. 2024ODICE: Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Updatecited by 21
  4. 2024Instruction-Guided Visual Maskingcited by 17
  5. 2023Mind the Gap: Offline Policy Optimization for Imperfect Rewardscited by 15
  6. 2024Offline Imitation Learning Through Graph Search and Retrievalcited by 14
  7. 2024MOTO: Offline Pre-training to Online Fine-tuning for Model-based Robot Learningcited by 13
  8. 2023CEIL: Generalized Contextual Imitation Learningcited by 12
  9. 2024Efficient Imitation Learning with Conservative World Modelscited by 11
  10. 2024Reward-free World Models for Online Imitation Learningcited by 7
  11. 2025Temporal Distance-aware Transition Augmentation for Offline Model-based Reinforcement Learningcited by 5
  12. 2024Context-Former: Stitching via Latent Conditioned Sequence Modelingcited by 3
5 more
  1. 2025MisoDICE: Multi-Agent Imitation from Unlabeled Mixed-Quality Demonstrationscited by 3
  2. 2025Robust Offline Imitation Learning Through State-level Trajectory Stitchingcited by 1
  3. 2025Restoring Noisy Demonstration for Imitation Learning With Diffusion Modelscited by 1
  4. 2025Coupled Distributional Random Expert Distillation for World Model Online Imitation Learningcited by 0
  5. 2026Latent Wasserstein Adversarial Imitation Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations
  2. 2025Learning What to Do and What Not To Do: Offline Imitation from Expert and Undesirable Demonstrations
  3. 2025Offline Imitation Learning upon Arbitrary Demonstrations by Pre-Training Dynamics Representations
  4. 2022Model-based Offline Imitation Learning with Non-expert Data
  5. 2024How to Leverage Diverse Demonstrations in Offline Imitation Learning
  6. 2024Offline Imitation Learning with Model-based Reverse Augmentation
  7. 2025Using Non-Expert Data to Robustify Imitation Learning via Offline Reinforcement Learning
  8. 2024SPRINQL: Sub-optimal Demonstrations driven Offline Imitation Learning