# Blindness of score-based methods to isolated components and mixing proportions

arXiv: https://arxiv.org/abs/2008.10087 · HTML: https://mlsanity.com/paper/2008.10087/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Aug 2020.

## Built on by (most cited first, 30 of 34)

- [Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models](https://mlsanity.com/paper/2310.13102/) (2023)
- [Kernel Stein Discrepancy Descent](https://mlsanity.com/paper/2105.09994/) (2021)
- [Robust Generalised Bayesian Inference for Intractable Likelihoods](https://mlsanity.com/paper/2104.07359/) (2021)
- [Can Push-forward Generative Models Fit Multimodal Distributions?](https://mlsanity.com/paper/2206.14476/) (2022)
- [KSD Aggregated Goodness-of-fit Test](https://mlsanity.com/paper/2202.00824/) (2022)
- [Diffusive Gibbs Sampling](https://mlsanity.com/paper/2402.03008/) (2024)
- [Composition and Control with Distilled Energy Diffusion Models and Sequential Monte Carlo](https://mlsanity.com/paper/2502.12786/) (2025)
- [Composite Goodness-of-fit Tests with Kernels](https://mlsanity.com/paper/2111.10275/) (2021)
- [Sample Complexity Bounds for Score-Matching: Causal Discovery and Generative Modeling](https://mlsanity.com/paper/2310.18123/) (2023)
- [Generalised Bayesian Inference for Discrete Intractable Likelihood](https://mlsanity.com/paper/2206.08420/) (2022)
- [Towards Healing the Blindness of Score Matching](https://mlsanity.com/paper/2209.07396/) (2022)
- [Controlling Moments with Kernel Stein Discrepancies](https://mlsanity.com/paper/2211.05408/) (2022)
- [Entropy-based Training Methods for Scalable Neural Implicit Sampler](https://mlsanity.com/paper/2306.04952/) (2023)
- [MixFlows: principled variational inference via mixed flows](https://mlsanity.com/paper/2205.07475/) (2022)
- [On the Wasserstein Gradient Flow Interpretation of Drifting Models](https://mlsanity.com/paper/2605.05118/) (2026)
- [Kernel Stein Discrepancy thinning: a theoretical perspective of pathologies and a practical fix with regularization](https://mlsanity.com/paper/2301.13528/) (2023)
- [Coin Sampling: Gradient-Based Bayesian Inference without Learning Rates](https://mlsanity.com/paper/2301.11294/) (2023)
- [Moment Matching Denoising Gibbs Sampling](https://mlsanity.com/paper/2305.11650/) (2023)
- [On the Robustness of Kernel Goodness-of-Fit Tests](https://mlsanity.com/paper/2408.05854/) (2024)
- [Diffusion-PINN Sampler](https://mlsanity.com/paper/2410.15336/) (2024)
- [Maximum Likelihood Learning of Unnormalized Models for Simulation-Based Inference](https://mlsanity.com/paper/2210.14756/) (2022)
- [A Practical Diffusion Path for Sampling](https://mlsanity.com/paper/2406.14040/) (2024)
- [A Unified View of Optimal Kernel Hypothesis Testing](https://mlsanity.com/paper/2503.07084/) (2025)
- [Using Perturbation to Improve Goodness-of-Fit Tests based on Kernelized Stein Discrepancy](https://mlsanity.com/paper/2304.14762/) (2023)
- [Benchmarking Bayesian neural networks and evaluation metrics for regression tasks](https://mlsanity.com/paper/2206.06779/) (2022)
- [A stable deep adversarial learning approach for geological facies generation](https://mlsanity.com/paper/2305.13318/) (2023)
- [Fast and Scalable Score-Based Kernel Calibration Tests](https://mlsanity.com/paper/2510.14711/) (2025)
- [Diffusion-based Annealed Boltzmann Generators : benefits, pitfalls and hopes](https://mlsanity.com/paper/2601.21026/) (2026)
- [Learning Energy-Based Models from Stochastic Interpolants using Spatiotemporal Differences](https://mlsanity.com/paper/2605.26850/) (2026)
- [Unifying Distributional Training for One-Step Visual Generation](https://mlsanity.com/paper/2609.35763/) (2026)

## Abstract (arXiv)

Statistical tasks such as density estimation and approximate Bayesian inference often involve densities with unknown normalising constants. Score-based methods, including score matching, are popular techniques as they are free of normalising constants. Although these methods enjoy theoretical guarantees, a little-known fact is that they exhibit practical failure modes when the unnormalised distribution of interest has isolated components -- they cannot discover isolated components or identify the correct mixing proportions between components. We demonstrate these findings using simple distributions and present heuristic attempts to address these issues. We hope to bring the attention of theoreticians and practitioners to these issues when developing new algorithms and applications.