# k-means as a variational EM approximation of Gaussian mixture models

arXiv: https://arxiv.org/abs/1704.04812 · HTML: https://mlsanity.com/paper/1704.04812/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2017.

## Built on by (most cited first, 8 of 8)

- [Prototypical Self-Explainable Models Without Re-training](https://mlsanity.com/paper/2312.07822/) (2023)
- [A sampling-based approach for efficient clustering in large datasets](https://mlsanity.com/paper/2112.14793/) (2021)
- [Semi-Supervised Clustering with Inaccurate Pairwise Annotations](https://mlsanity.com/paper/2104.02146/) (2021)
- [Fisher Information Embedding for Node and Graph Learning](https://mlsanity.com/paper/2305.07580/) (2023)
- [Clustered FedStack: Intermediate Global Models with Bayesian Information Criterion](https://mlsanity.com/paper/2309.11044/) (2023)
- [Color Counting for Fashion, Art, and Design](https://mlsanity.com/paper/2110.06682/) (2021)
- [Sublinear Variational Optimization of Gaussian Mixture Models with Millions to Billions of Parameters](https://mlsanity.com/paper/2501.12299/) (2025)
- [K-Means as a Radial Basis function Network: a Variational and Gradient-based Equivalence](https://mlsanity.com/paper/2603.04625/) (2026)

## Abstract (arXiv)

We show that k-means (Lloyd's algorithm) is obtained as a special case when truncated variational EM approximations are applied to Gaussian Mixture Models (GMM) with isotropic Gaussians. In contrast to the standard way to relate k-means and GMMs, the provided derivation shows that it is not required to consider Gaussians with small variances or the limit case of zero variances. There are a number of consequences that directly follow from our approach: (A) k-means can be shown to increase a free energy associated with truncated distributions and this free energy can directly be reformulated in terms of the k-means objective; (B) k-means generalizations can directly be derived by considering the 2nd closest, 3rd closest etc. cluster in addition to just the closest one; and (C) the embedding of k-means into a free energy framework allows for theoretical interpretations of other k-means generalizations in the literature. In general, truncated variational EM provides a natural and rigorous quantitative link between k-means-like clustering and GMM clustering algorithms which may be very relevant for future theoretical and empirical studies.