mlsanityData to 14 Oct 2026

Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow

arXiv 1810.00821 ↗First posted 1 Oct 2018cs.LG stat.MLcited by 105

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Adversarial learning methods have been proposed for a wide range of applications, but the training of adversarial models can be notoriously unstable. Effectively balancing the performance of the generator and discriminator is critical, since a discriminator that achieves very high accuracy will produce relatively uninformative gradients. In this work, we propose a simple and general technique to constrain information flow in the discriminator by means of an information bottleneck. By enforcing a constraint on the mutual information between the observations and the discriminator's internal representation, we can effectively modulate the discriminator's accuracy and maintain useful and informative gradients. We demonstrate that our proposed variational discriminator bottleneck (VDB) leads to significant improvements across three distinct application areas for adversarial learning algorithms. Our primary evaluation studies the applicability of the VDB to imitation learning of dynamic continuous control skills, such as running. We show that our method can learn such skills directly from raw video demonstrations, substantially outperforming prior adversarial imitation learning methods. The VDB can also be combined with adversarial inverse reinforcement learning to learn parsimonious reward functions that can be transferred and re-optimized in new settings. Finally, we demonstrate that VDB can train GANs more effectively for image generation, improving upon a number of prior stabilization methods.

Key facts. Cited by 105 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 31 of its 50 references that are in the map. First posted on arXiv 1 Oct 2018.

Builds on

31 of 50 references.

References · 31

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Auto-Encoding Variational Bayescited by 4,053
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  5. 2016Improved Techniques for Training GANscited by 2,970
  6. 2015Trust Region Policy Optimizationcited by 2,580
  7. 2017Improved Training of Wasserstein GANscited by 2,235
  8. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  9. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  10. 2016Generative Adversarial Imitation Learningcited by 1,597
  11. 2016Deep Variational Information Bottleneckcited by 930
  12. 2015Deep Learning and the Information Bottleneck Principlecited by 802
19 more
  1. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  2. 2018Which Training Methods for GANs do actually Converge?cited by 679
  3. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  4. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 464
  5. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  6. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
  7. 2016Generating Videos with Scene Dynamicscited by 376
  8. 2015Adversarial Autoencoderscited by 364
  9. 2017Are GANs Created Equal? A Large-Scale Studycited by 267
  10. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 168
  11. 2016Information Dropout: Learning Optimal Representations Through Noisy Computationcited by 132
  12. 2016Mode Regularized Generative Adversarial Networkscited by 123
  13. 2017BEGAN: Boundary Equilibrium Generative Adversarial Networkscited by 121
  14. 2016Energy-based Generative Adversarial Networkcited by 119
  15. 2017Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networkscited by 107
  16. 2017Learning human behaviors from motion capture by adversarial imitationcited by 77
  17. 2016Amortised MAP Inference for Image Super-resolutioncited by 68
  18. 2017On Convergence and Stability of GANscited by 18
  19. 2017Symmetric Variational Autoencoder and Connections to Adversarial Learningcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 10; 2020: 5; 2021: 19; 2022: 16; 2023: 18; 2024: 15; 2025: 13; 2026: 8.

105 papers.

Sort:

By area:

  1. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 221
  2. 2021Adversarial Graph Augmentation to Improve Graph Contrastive Learningcited by 200
  3. 2019Imitation Learning as f-Divergence Minimizationcited by 123
  4. 2021Off-Policy Imitation Learning from Observationscited by 70
  5. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variablescited by 50
  6. 2021Farewell to Mutual Information: Variational Distillation for Cross-Modal Person Re-Identificationcited by 50
  7. 2022Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representationscited by 48
  8. 2022Vision-Based Manipulators Need to Also See from Their Handscited by 47
  9. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreementcited by 46
  10. 2019Task-Relevant Adversarial Imitation Learningcited by 45
  11. 2021Improving Subgraph Recognition with Variational Graph Information Bottleneckcited by 45
  12. 2019Positive-Unlabeled Reward Learningcited by 37
38 more
  1. 2020Intrinsic Reward Driven Imitation Learning via Generative Modelcited by 35
  2. 2023LocoMuJoCo: A Comprehensive Imitation Learning Benchmark for Locomotioncited by 33
  3. 2023LS-IQ: Implicit Reward Regularization for Inverse Reinforcement Learningcited by 29
  4. 2021Recognizing Predictive Substructures with Subgraph Information Bottleneckcited by 28
  5. 2022Optimal-er Auctions through Attentioncited by 27
  6. 2018Inverse reinforcement learning for video gamescited by 26
  7. 2021Online Multi-Granularity Distillation for GAN Compressioncited by 24
  8. 2021Domain-Robust Visual Imitation Learning with Mutual Information Constraintscited by 20
  9. 2021VoteHMR: Occlusion-Aware Voting Network for Robust 3D Human Mesh Recovery from Partial Point Cloudscited by 18
  10. 2023RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictabilitycited by 18
  11. 2021Navigation Turing Test (NTT): Learning to Evaluate Human-Like Navigationcited by 17
  12. 2022Discriminator-Guided Model-Based Offline Imitation Learningcited by 17
  13. 2024Off-Dynamics Reinforcement Learning via Domain Adaptation and Reward Augmented Imitationcited by 17
  14. 2024A Survey of Imitation Learning Methods, Environments and Metricscited by 16
  15. 2020Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learningcited by 15
  16. 2021Drop-Bottleneck: Learning Discrete Compressed Representation for Noise-Robust Explorationcited by 15
  17. 2021Behavior-Driven Synthesis of Human Dynamicscited by 14
  18. 2022Bayesian Imitation Learning for End-to-End Mobile Manipulationcited by 14
  19. 2020Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Explorationcited by 13
  20. 2021Imitation by Predicting Observationscited by 13
  21. 2024Inf2Guard: An Information-Theoretic Framework for Learning Privacy-Preserving Representations against Inference Attackscited by 13
  22. 2019VILD: Variational Imitation Learning with Diverse-quality Demonstrationscited by 12
  23. 2023Reward-Consistent Dynamics Models are Strongly Generalizable for Offline Reinforcement Learningcited by 12
  24. 2022Imitation Learning from Observations under Transition Model Disparitycited by 11
  25. 2023SeMAIL: Eliminating Distractors in Visual Imitation via Separated Modelscited by 10
  26. 2023Learning High-Order Relationships of Brain Regionscited by 10
  27. 2021Neuron Campaign for Initialization Guided by Information Bottleneck Theorycited by 9
  28. 2023Information Design in Multi-Agent Reinforcement Learningcited by 9
  29. 2024Self-Explainable Temporal Graph Networks based on Graph Information Bottleneckcited by 9
  30. 2025Mitigating Hallucinations in Large Vision-Language Models by Adaptively Constraining Information Flowcited by 9
  31. 2020Off-Policy Adversarial Inverse Reinforcement Learningcited by 8
  32. 2021Cross Domain Robot Imitation with Invariant Representationcited by 8
  33. 2024A Comparison of Imitation Learning Algorithms for Bimanual Manipulationcited by 8
  34. 2022Goal-Aware Generative Adversarial Imitation Learning from Imperfect Demonstration for Robotic Cloth Manipulationcited by 7
  35. 2019Imitation Learning from Pixel-Level Demonstrations by HashRewardcited by 6
  36. 2021Grounded Relational Inference: Domain Knowledge Driven Explainable Autonomous Drivingcited by 6
  37. 2022Disentangled Action Recognition with Knowledge Basescited by 6
  38. 2021Automatic Curricula via Expert Demonstrationscited by 5

50 most cited of 105 All 105 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2020Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization
  2. 2021Deterministic and Discriminative Imitation (D2-Imitation): Revisiting Adversarial Imitation for Sample Efficiency
  3. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning
  4. 2024Diffusing States and Matching Scores: A New Framework for Imitation Learning
  5. 2021Visual Adversarial Imitation Learning using Variational Models
  6. 2026FAIL: Flow Matching Adversarial Imitation Learning for Image Generation
  7. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets
  8. 2023Look ATME: The Discriminator Mean Entropy Needs Attention