# On Learning Prediction-Focused Mixtures

arXiv: https://arxiv.org/abs/2110.13221 · HTML: https://mlsanity.com/paper/2110.13221/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 20 references that are in the map.
- First posted on arXiv 25 Oct 2021.

## Builds on (most cited first, 7 of 7)

- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [POPCORN: Partially Observed Prediction COnstrained ReiNforcement Learning](https://mlsanity.com/paper/2001.04032/) (2020)
- [Loss-Calibrated Approximate Inference in Bayesian Neural Networks](https://mlsanity.com/paper/1805.03901/) (2018)
- [On the Difference Between the Information Bottleneck and the Deep Information Bottleneck](https://mlsanity.com/paper/1912.13480/) (2019)
- [Prediction Focused Topic Models via Feature Selection](https://mlsanity.com/paper/1910.05495/) (2019)
- [Prediction-Constrained Training for Semi-Supervised Mixture and Topic Models](https://mlsanity.com/paper/1707.07341/) (2017)

## Built on by (most cited first, 3 of 3)

- [Solver-Free Decision-Focused Learning for Linear Optimization Problems](https://mlsanity.com/paper/2505.22224/) (2025)
- [A Joint Learning Approach for Semi-supervised Neural Topic Modeling](https://mlsanity.com/paper/2204.03208/) (2022)
- [Decision-Focused Model-based Reinforcement Learning for Reward Transfer](https://mlsanity.com/paper/2304.03365/) (2023)

## Abstract (arXiv)

Probabilistic models help us encode latent structures that both model the data and are ideally also useful for specific downstream tasks. Among these, mixture models and their time-series counterparts, hidden Markov models, identify discrete components in the data. In this work, we focus on a constrained capacity setting, where we want to learn a model with relatively few components (e.g. for interpretability purposes). To maintain prediction performance, we introduce prediction-focused modeling for mixtures, which automatically selects the dimensions relevant to the prediction task. Our approach identifies relevant signal from the input, outperforms models that are not prediction-focused, and is easy to optimize; we also characterize when prediction-focused modeling can be expected to work.