# ExpertNet: A Symbiosis of Classification and Clustering

arXiv: https://arxiv.org/abs/2201.06344 · HTML: https://mlsanity.com/paper/2201.06344/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 43 references that are in the map.
- First posted on arXiv 17 Jan 2022.

## Builds on (most cited first, 13 of 13)

- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [Size-Independent Sample Complexity of Neural Networks](https://mlsanity.com/paper/1712.06541/) (2017)
- [Semi-Supervised Learning with Ladder Networks](https://mlsanity.com/paper/1507.02672/) (2015)
- [Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering](https://mlsanity.com/paper/1610.04794/) (2016)
- [Learning Internal Representations (COLT 1995)](https://mlsanity.com/paper/1911.05781/) (2019)
- [Learning Tasks for Multitask Learning: Heterogenous Patient Populations in the ICU](https://mlsanity.com/paper/1806.02878/) (2018)
- [Stacked What-Where Auto-encoders](https://mlsanity.com/paper/1506.02351/) (2015)
- [Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification](https://mlsanity.com/paper/1606.06582/) (2016)
- [Mimic and Conquer: Heterogeneous Tree Structure Distillation for Syntactic NLP](https://mlsanity.com/paper/2009.07411/) (2020)
- [On Classifying Sepsis Heterogeneity in the ICU: Insight Using Machine Learning](https://mlsanity.com/paper/1912.00672/) (2019)

## Abstract (arXiv)

A widely used paradigm to improve the generalization performance of high-capacity neural models is through the addition of auxiliary unsupervised tasks during supervised training. Tasks such as similarity matching and input reconstruction have been shown to provide a beneficial regularizing effect by guiding representation learning. Real data often has complex underlying structures and may be composed of heterogeneous subpopulations that are not learned well with current approaches. In this work, we design ExpertNet, which uses novel training strategies to learn clustered latent representations and leverage them by effectively combining cluster-specific classifiers. We theoretically analyze the effect of clustering on its generalization gap, and empirically show that clustered latent representations from ExpertNet lead to disentangling the intrinsic structure and improvement in classification performance. ExpertNet also meets an important real-world need where classifiers need to be tailored for distinct subpopulations, such as in clinical risk models. We demonstrate the superiority of ExpertNet over state-of-the-art methods on 6 large clinical datasets, where our approach leads to valuable insights on group-specific risks.