# Self-regularizing Property of Nonparametric Maximum Likelihood Estimator in Mixture Models

arXiv: https://arxiv.org/abs/2008.08244 · HTML: https://mlsanity.com/paper/2008.08244/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Aug 2020.

## Built on by (most cited first, 17 of 17)

- [Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flow](https://mlsanity.com/paper/2301.01766/) (2023)
- [Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixtures](https://mlsanity.com/paper/2504.05300/) (2025)
- [A Mean Field Approach to Empirical Bayes Estimation in High-dimensional Linear Regression](https://mlsanity.com/paper/2309.16843/) (2023)
- [Robust Model Selection and Nearly-Proper Learning for GMMs](https://mlsanity.com/paper/2106.02774/) (2021)
- [On the best approximation by finite Gaussian mixtures](https://mlsanity.com/paper/2404.08913/) (2024)
- [Nonparametric MLE for Gaussian Location Mixtures: Certified Computation and Generic Behavior](https://mlsanity.com/paper/2503.20193/) (2025)
- [Fisher-Pitman permutation tests based on nonparametric Poisson mixtures with application to single cell genomics](https://mlsanity.com/paper/2106.03022/) (2021)
- [When Can Voting Help, Hurt, or Change Course? Exact Structure of Binary Test-Time Aggregation](https://mlsanity.com/paper/2605.05592/) (2026)
- [Entropy regularization in probabilistic clustering](https://mlsanity.com/paper/2307.10065/) (2023)
- [Neural-g: A Deep Learning Framework for Mixing Density Estimation](https://mlsanity.com/paper/2406.05986/) (2024)
- [Model-free Estimation of Latent Structure via Multiscale Nonparametric Maximum Likelihood](https://mlsanity.com/paper/2410.22248/) (2024)
- [EM Approaches to Nonparametric Estimation for Mixture of Linear Regressions](https://mlsanity.com/paper/2510.14890/) (2025)
- [Gaussian mixtures and non-parametric likelihoods through the lens of statistical mechanics](https://mlsanity.com/paper/2603.23196/) (2026)
- [Empirical Bayes Rebiasing](https://mlsanity.com/paper/2605.08069/) (2026)
- [Density estimation for Hellinger via minimum-distance estimators: mixtures of Gaussians, log-concave, and more](https://mlsanity.com/paper/2606.11469/) (2026)
- [Empirical Bayes for correlated Gaussian sequence model](https://mlsanity.com/paper/2607.03596/) (2026)
- [Polylogarithmic Sparsity of Randomly Reweighted NPMLEs for Gaussian Mixtures](https://mlsanity.com/paper/2610.01088/) (2026)

## Abstract (arXiv)

Introduced by Kiefer and Wolfowitz citeKW56, the nonparametric maximum likelihood estimator (NPMLE) is a widely used methodology for learning mixture odels and empirical Bayes estimation. Sidestepping the non-convexity in mixture likelihood, the NPMLE estimates the mixing distribution by maximizing the total likelihood over the space of probability measures, which can be viewed as an extreme form of overparameterization. In this paper we discover a surprising property of the NPMLE solution. Consider, for example, a Gaussian mixture model on the real line with a subgaussian mixing distribution. Leveraging complex-analytic techniques, we show that with high probability the NPMLE based on a sample of size n has O(log n) atoms (mass points), significantly improving the deterministic upper bound of n due to Lindsay citelindsay1983geometry1. Notably, any such Gaussian mixture is statistically indistinguishable from a finite one with O(log n) components (and this is tight for certain mixtures). Thus, absent any explicit form of model selection, NPMLE automatically chooses the right model complexity, a property we term self-regularization. Extensions to other exponential families are given. As a statistical application, we show that this structural property can be harnessed to bootstrap existing Hellinger risk bound of the (parametric) MLE for finite Gaussian mixtures to the NPMLE for general Gaussian mixtures, recovering a result of Zhang citezhang2009generalized.