# Minimal Achievable Sufficient Statistic Learning

arXiv: https://arxiv.org/abs/1905.07822 · HTML: https://mlsanity.com/paper/1905.07822/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2019.

## Built on by (most cited first, 6 of 6)

- [Examining and Combating Spurious Features under Distribution Shift](https://mlsanity.com/paper/2106.07171/) (2021)
- [Lossy Compression for Lossless Prediction](https://mlsanity.com/paper/2106.10800/) (2021)
- [Gaussian Embeddings: How JEPAs Secretly Learn Your Data Density](https://mlsanity.com/paper/2510.05949/) (2025)
- [Learning Summary Statistics for Bayesian Inference with Autoencoders](https://mlsanity.com/paper/2201.12059/) (2022)
- [Self-Supervised Metric Learning in Multi-View Data: A Downstream Task Perspective](https://mlsanity.com/paper/2106.07138/) (2021)
- [Super-Resolution through StyleGAN Regularized Latent Search: A Realism-Fidelity Trade-off](https://mlsanity.com/paper/2311.16923/) (2023)

## Abstract (arXiv)

We introduce Minimal Achievable Sufficient Statistic (MASS) Learning, a training method for machine learning models that attempts to produce minimal sufficient statistics with respect to a class of functions (e.g. deep networks) being optimized over. In deriving MASS Learning, we also introduce Conserved Differential Information (CDI), an information-theoretic quantity that - unlike standard mutual information - can be usefully applied to deterministically-dependent continuous random variables like the input and output of a deep network. In a series of experiments, we show that deep networks trained with MASS Learning achieve competitive performance on supervised learning and uncertainty quantification benchmarks.