mlsanityData to 7 Oct 2026

Blindness of score-based methods to isolated components and mixing proportions

arXiv 2008.10087 ↗First posted 23 Aug 2020stat.ML cs.LGcited by 33

Not on the map. Why?

Abstract (arXiv)

Statistical tasks such as density estimation and approximate Bayesian inference often involve densities with unknown normalising constants. Score-based methods, including score matching, are popular techniques as they are free of normalising constants. Although these methods enjoy theoretical guarantees, a little-known fact is that they exhibit practical failure modes when the unnormalised distribution of interest has isolated components -- they cannot discover isolated components or identify the correct mixing proportions between components. We demonstrate these findings using simple distributions and present heuristic attempts to address these issues. We hope to bring the attention of theoreticians and practitioners to these issues when developing new algorithms and applications.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 8; 2023: 8; 2024: 4; 2025: 5; 2026: 5.

33 papers.

Sort:

By area:

  1. 2023Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Modelscited by 65
  2. 2021Kernel Stein Discrepancy Descentcited by 57
  3. 2021Robust Generalised Bayesian Inference for Intractable Likelihoodscited by 53
  4. 2022Can Push-forward Generative Models Fit Multimodal Distributions?cited by 41
  5. 2022KSD Aggregated Goodness-of-fit Testcited by 21
  6. 2024Diffusive Gibbs Samplingcited by 21
  7. 2025Composition and Control with Distilled Energy Diffusion Models and Sequential Monte Carlocited by 18
  8. 2021Composite Goodness-of-fit Tests with Kernelscited by 16
  9. 2023Sample Complexity Bounds for Score-Matching: Causal Discovery and Generative Modelingcited by 15
  10. 2022Generalised Bayesian Inference for Discrete Intractable Likelihoodcited by 14
  11. 2022Towards Healing the Blindness of Score Matchingcited by 13
  12. 2022Controlling Moments with Kernel Stein Discrepanciescited by 13
21 more
  1. 2023Entropy-based Training Methods for Scalable Neural Implicit Samplercited by 13
  2. 2022MixFlows: principled variational inference via mixed flowscited by 11
  3. 2023Kernel Stein Discrepancy thinning: a theoretical perspective of pathologies and a practical fix with regularizationcited by 9
  4. 2023Coin Sampling: Gradient-Based Bayesian Inference without Learning Ratescited by 8
  5. 2023Moment Matching Denoising Gibbs Samplingcited by 7
  6. 2024On the Robustness of Kernel Goodness-of-Fit Testscited by 7
  7. 2024Diffusion-PINN Samplercited by 7
  8. 2026On the Wasserstein Gradient Flow Interpretation of Drifting Modelscited by 6
  9. 2022Maximum Likelihood Learning of Unnormalized Models for Simulation-Based Inferencecited by 5
  10. 2024A Practical Diffusion Path for Samplingcited by 5
  11. 2023Using Perturbation to Improve Goodness-of-Fit Tests based on Kernelized Stein Discrepancycited by 4
  12. 2025A Unified View of Optimal Kernel Hypothesis Testingcited by 4
  13. 2022Benchmarking Bayesian neural networks and evaluation metrics for regression taskscited by 3
  14. 2023A stable deep adversarial learning approach for geological facies generationcited by 2
  15. 2025Fast and Scalable Score-Based Kernel Calibration Testscited by 2
  16. 2026Learning Energy-Based Models from Stochastic Interpolants using Spatiotemporal Differencescited by 2
  17. 2026Diffusion-based Annealed Boltzmann Generators : benefits, pitfalls and hopescited by 1
  18. 2026Unifying Distributional Training for One-Step Visual Generationcited by 1
  19. 2025Correcting Mode Proportion Bias in Generalized Bayesian Inference via a Weighted Kernel Stein Discrepancycited by 0
  20. 2025Energy Guided Geometric Flow Matchingcited by 0
  21. 2026Error-Corrected Inference-Time Scaling for Imperfect Diffusion Modelscited by 0