mlsanityData to 7 Oct 2026

Pseudo-Spherical Contrastive Divergence

arXiv 2111.00780 ↗First posted 1 Nov 2021cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Energy-based models (EBMs) offer flexible distribution parametrization. However, due to the intractable partition function, they are typically trained via contrastive divergence for maximum likelihood estimation. In this paper, we propose pseudo-spherical contrastive divergence (PS-CD) to generalize maximum likelihood learning of EBMs. PS-CD is derived from the maximization of a family of strictly proper homogeneous scoring rules, which avoids the computation of the intractable partition function and provides a generalized family of learning objectives that include contrastive divergence as a special case. Moreover, PS-CD allows us to flexibly choose various learning objectives to train EBMs without additional computational cost or variational minimax optimization. Theoretical analysis on the proposed method and extensive experiments on both synthetic data and commonly used image datasets demonstrate the effectiveness and modeling flexibility of PS-CD, as well as its robustness to data contamination, thus showing its superiority over maximum likelihood and f-EBMs.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 36 of its 89 references that are in the map. First posted on arXiv 1 Nov 2021.

Builds on

36 of 89 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  5. 2016Improved Techniques for Training GANscited by 2,938
  6. 2013Describing Textures in the Wildcited by 2,110
  7. 2016Density estimation using Real NVPcited by 2,029
  8. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  9. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  10. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  11. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  12. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
24 more
  1. 2018Which Training Methods for GANs do actually Converge?cited by 672
  2. 2017Wasserstein GANcited by 635
  3. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 504
  4. 2015Importance Weighted Autoencoderscited by 454
  5. 2020Reliable Fidelity and Diversity Metrics for Generative Modelscited by 425
  6. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 402
  7. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 381
  8. 2015MADE: Masked Autoencoder for Distribution Estimationcited by 306
  9. 2019Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Modelcited by 147
  10. 2016Stochastic Variance Reduction for Nonconvex Optimizationcited by 143
  11. 2016R'enyi Divergence Variational Inferencecited by 104
  12. 2018Bias and Generalization in Deep Generative Models: An Empirical Studycited by 83
  13. 2019Flow Contrastive Estimation of Energy-Based Modelscited by 75
  14. 2020No MCMC for me: Amortized sampling for fast and stable training of energy-based modelscited by 64
  15. 2017Sub-sampled Cubic Regularization for Non-convex Optimizationcited by 63
  16. 2019Maximum Entropy Generators for Energy-Based Modelscited by 55
  17. 2019Model Based Planning with Energy Based Modelscited by 32
  18. 2019Exponential Family Estimation via Adversarial Dynamics Embeddingcited by 30
  19. 2020Training Deep Energy-Based Models with f-Divergence Minimizationcited by 29
  20. 2019Bridging the Gap Between f-GANs and Wasserstein GANscited by 23
  21. 2018Stochastic Gradient Descent with Biased but Consistent Gradient Estimatorscited by 18
  22. 2018Kernel Exponential Family Estimation via Doubly Dual Embeddingcited by 16
  23. 2017Unbiased Markov chain Monte Carlo with couplingscited by 10
  24. 2015BlackOut: Speeding up Recurrent Neural Network Language Models With Very Large Vocabulariescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 2; 2025: 1; 2026: 1.

7 papers.

  1. 2025Proper scoring rules for estimation and forecast evaluationcited by 18
  2. 2023Guiding Energy-based Models via Contrastive Latent Variablescited by 17
  3. 2023Energy-Based Models for Anomaly Detection: A Manifold Diffusion Recovery Approachcited by 17
  4. 2022RenyiCL: Contrastive Representation Learning with Skew Renyi Divergencecited by 8
  5. 2024Language Generation with Strictly Proper Scoring Rulescited by 8
  6. 2024Unbiased Estimating Equation on Inverse Divergence and Its Conditionscited by 0
  7. 2026Projective Maximum Entropy: Universality and Acceptance-Region Calibrationcited by 0