Data Interpolants -- That's What Discriminators in Higher-order Gradient-regularized GANs Are
Not on the map. Why?
Abstract (arXiv)
We consider the problem of optimizing the discriminator in generative adversarial networks (GANs) subject to higher-order gradient regularization. We show analytically, via the least-squares (LSGAN) and Wasserstein (WGAN) GAN variants, that the discriminator optimization problem is one of interpolation in n-dimensions. The optimal discriminator, derived using variational Calculus, turns out to be the solution to a partial differential equation involving the iterated Laplacian or the polyharmonic operator. The solution is implementable in closed-form via polyharmonic radial basis function (RBF) interpolation. In view of the polyharmonic connection, we refer to the corresponding GANs as Poly-LSGAN and Poly-WGAN. Through experimental validation on multivariate Gaussians, we show that implementing the optimal RBF discriminator in closed-form, with penalty orders m ≈lceil fracn2 rceil , results in superior performance, compared to training GAN with arbitrarily chosen discriminator architectures. We employ the Poly-WGAN discriminator to model the latent space distribution of the data with encoder-decoder-based GAN flavors such as Wasserstein autoencoders.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 48 of its 75 references that are in the map. First posted on arXiv 1 Jun 2023.
Builds on
48 of 75 references.
References · 48
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
- 2022Generative Adversarial Networkscited by 5,627
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
- 2014Deep Learning Face Attributes in the Wildcited by 3,693
- 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
- 2017Improved Training of Wasserstein GANscited by 2,235
- 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
36 more
- 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
- 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
- 2018Demystifying MMD GANscited by 1,293
- 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
- 2021Alias-Free Generative Adversarial Networkscited by 1,184
- 2016Least Squares Generative Adversarial Networkscited by 1,001
- 2018Which Training Methods for GANs do actually Converge?cited by 679
- 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 664
- 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
- 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 506
- 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 464
- 2017Wasserstein Auto-Encoderscited by 373
- 2015Generative Moment Matching Networkscited by 257
- 2017MMD GAN: Towards Deeper Understanding of Moment Matching Networkcited by 254
- 2017Training GANs with Optimismcited by 217
- 2017The Cramer Distance as a Solution to Biased Wasserstein Gradientscited by 119
- 2017Stabilizing Training of Generative Adversarial Networks through Regularizationcited by 118
- 2018How Well Generative Adversarial Networks Learn Distributionscited by 78
- 2017On the regularization of Wasserstein GANscited by 65
- 2019Learning Generative Models across Incomparable Spacescited by 53
- 2021On Aliased Resizing and Surprising Subtleties in GAN Evaluationcited by 51
- 2017Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Stepcited by 45
- 2020A case for new neural network smoothness constraintscited by 43
- 2021KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Supportcited by 37
- 2019Adversarial Lipschitz Regularizationcited by 30
- 2018On gradient regularizers for MMD GANscited by 28
- 2018Improving MMD-GAN Training with Repulsive Loss Functioncited by 22
- 2020On the Convergence of Gradient Descent in GANs: MMD GAN As a Gradient Flowcited by 21
- 2021A Neural Tangent Kernel Perspective of GANscited by 21
- 2022Kantorovich Strikes Back! Wasserstein GANs are not Optimal Transport?cited by 19
- 2020Unbalanced Sobolev Descentcited by 11
- 2018Cramer-Wold AutoEncodercited by 10
- 2020Flow Field Reconstructions with GANs based on Radial Basis Functionscited by 7
- 2017On the Limitations of First-Order Approximation in GAN Dynamicscited by 4
- 2019Adversarial Training is a Form of Data-dependent Operator Norm Regularizationcited by 4
- 2022Single-level Adversarial Data Synthesis based on Neural Tangent Kernelscited by 2
Built on by
Citing papers per year: 2023: 2.
2 papers.