# Learning Implicit Generative Models by Teaching Explicit Ones

arXiv: https://arxiv.org/abs/1807.03870 · HTML: https://mlsanity.com/paper/1807.03870/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2018.

## Built on by (most cited first, 3 of 3)

- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [Assessing the ability of generative adversarial networks to learn canonical medical image statistics](https://mlsanity.com/paper/2204.12007/) (2022)
- [Deep Generative Modeling on Limited Data with Regularization by Nontransferable Pre-trained Models](https://mlsanity.com/paper/2208.14133/) (2022)

## Abstract (arXiv)

Implicit generative models are difficult to train as no explicit density functions are defined. Generative adversarial nets (GANs) present a minimax framework to train such models, which however can suffer from mode collapse due to the nature of the JS-divergence. This paper presents a learning by teaching (LBT) approach to learning implicit models, which intrinsically avoids the mode collapse problem by optimizing a KL-divergence rather than the JS-divergence in GANs. In LBT, an auxiliary density estimator is introduced to fit the implicit model's distribution while the implicit model teaches the density estimator to match the data distribution. LBT is formulated as a bilevel optimization problem, whose optimal generator matches the true data distribution. LBT can be naturally integrated with GANs to derive a hybrid LBT-GAN that enjoys complimentary benefits. Finally, we present a stochastic gradient ascent algorithm with unrolling to solve the challenging learning problems. Experimental results demonstrate the effectiveness of our method.