# Deep clustering: On the link between discriminative models and K-means

arXiv: https://arxiv.org/abs/1810.04246 · HTML: https://mlsanity.com/paper/1810.04246/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2018.

## Built on by (most cited first, 30 of 37)

- [Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation](https://mlsanity.com/paper/2110.04202/) (2021)
- [SPICE: Semantic Pseudo-labeling for Image Clustering](https://mlsanity.com/paper/2103.09382/) (2021)
- [A Comprehensive Survey on Source-free Domain Adaptation](https://mlsanity.com/paper/2302.11803/) (2023)
- [Source-Free Domain Adaptation for Image Segmentation](https://mlsanity.com/paper/2108.03152/) (2021)
- [Attention-driven Graph Clustering Network](https://mlsanity.com/paper/2108.05499/) (2021)
- [Parametric Information Maximization for Generalized Category Discovery](https://mlsanity.com/paper/2212.00334/) (2022)
- [Unsupervised Domain Adaptation of Black-Box Source Models](https://mlsanity.com/paper/2101.02839/) (2021)
- [Pseudo-supervised Deep Subspace Clustering](https://mlsanity.com/paper/2104.03531/) (2021)
- [A Strong Baseline for Generalized Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2211.14126/) (2022)
- [ClusT3: Information Invariant Test-Time Training](https://mlsanity.com/paper/2310.12345/) (2023)
- [Reducing the Covariate Shift by Mirror Samples in Cross Domain Alignment](https://mlsanity.com/paper/2110.06448/) (2021)
- [Deep Attention-guided Graph Clustering with Dual Self-supervision](https://mlsanity.com/paper/2111.05548/) (2021)
- [Trust your Good Friends: Source-free Domain Adaptation by Reciprocal Neighborhood Clustering](https://mlsanity.com/paper/2309.00528/) (2023)
- [Large-Scale Hyperspectral Image Clustering Using Contrastive Learning](https://mlsanity.com/paper/2111.07945/) (2021)
- [Generalised Mutual Information for Discriminative Clustering](https://mlsanity.com/paper/2210.06300/) (2022)
- [Unsupervised Domain Adaptation via Distilled Discriminative Clustering](https://mlsanity.com/paper/2302.11984/) (2023)
- [Step-wise Distribution Alignment Guided Style Prompt Tuning for Source-free Cross-domain Few-shot Learning](https://mlsanity.com/paper/2411.10070/) (2024)
- [Deep Clustering With Consensus Representations](https://mlsanity.com/paper/2210.07063/) (2022)
- [Semantic Anchor Transport: Robust Test-Time Adaptation for Vision-Language Models](https://mlsanity.com/paper/2411.17002/) (2024)
- [Training Heterogeneous Features in Sequence to Sequence Tasks: Latent Enhanced Multi-filter Seq2Seq Model](https://mlsanity.com/paper/2105.08840/) (2021)
- [Deep Embedding Clustering Driven by Sample Stability](https://mlsanity.com/paper/2401.15989/) (2024)
- [Mutual-Information Based Few-Shot Classification](https://mlsanity.com/paper/2106.12252/) (2021)
- [Enhancing the Analysis of Software Failures in Cloud Computing Systems with Deep Learning](https://mlsanity.com/paper/2106.15182/) (2021)
- [Task Adaptive Feature Transformation for One-Shot Learning](https://mlsanity.com/paper/2304.06832/) (2023)
- [Generalised Mutual Information: a Framework for Discriminative Clustering](https://mlsanity.com/paper/2309.02858/) (2023)
- [An Adaptive Resonance Theory-based Topological Clustering Algorithm with a Self-Adjusting Vigilance Parameter](https://mlsanity.com/paper/2511.17983/) (2025)
- [Deep Contrastive Patch-Based Subspace Learning for Camera Image Signal Processing](https://mlsanity.com/paper/2104.00253/) (2021)
- [fMBN-E: Efficient Unsupervised Network Structure Ensemble and Selection for Clustering](https://mlsanity.com/paper/2107.02071/) (2021)
- [Self-Enhancing Multi-filter Sequence-to-Sequence Model](https://mlsanity.com/paper/2109.12399/) (2021)
- [Simplex Clustering via sBeta with Applications to Online Adjustment of Black-Box Predictions](https://mlsanity.com/paper/2208.00287/) (2022)

## Abstract (arXiv)

In the context of recent deep clustering studies, discriminative models dominate the literature and report the most competitive performances. These models learn a deep discriminative neural network classifier in which the labels are latent. Typically, they use multinomial logistic regression posteriors and parameter regularization, as is very common in supervised learning. It is generally acknowledged that discriminative objective functions (e.g., those based on the mutual information or the KL divergence) are more flexible than generative approaches (e.g., K-means) in the sense that they make fewer assumptions about the data distributions and, typically, yield much better unsupervised deep learning results. On the surface, several recent discriminative models may seem unrelated to K-means. This study shows that these models are, in fact, equivalent to K-means under mild conditions and common posterior models and parameter regularization. We prove that, for the commonly used logistic regression posteriors, maximizing the L2 regularized mutual information via an approximate alternating direction method (ADM) is equivalent to a soft and regularized K-means loss. Our theoretical analysis not only connects directly several recent state-of-the-art discriminative models to K-means, but also leads to a new soft and regularized deep K-means algorithm, which yields competitive performance on several image clustering benchmarks.