# Learning Structured Gaussians to Approximate Deep Ensembles

arXiv: https://arxiv.org/abs/2203.15485 · HTML: https://mlsanity.com/paper/2203.15485/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 29 Mar 2022.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Snapshot Ensembles: Train 1, get M for free](https://mlsanity.com/paper/1704.00109/) (2017)
- [An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution](https://mlsanity.com/paper/1807.03247/) (2018)
- [On the uncertainty of self-supervised monocular depth estimation](https://mlsanity.com/paper/2005.06209/) (2020)
- [Ensemble Distribution Distillation](https://mlsanity.com/paper/1905.00076/) (2019)
- [Structured Uncertainty Prediction Networks](https://mlsanity.com/paper/1802.07079/) (2018)
- [Fast, Exact and Multi-Scale Inference for Semantic Image Segmentation with Deep Gaussian CRFs](https://mlsanity.com/paper/1603.08358/) (2016)
- [Generating and Exploiting Probabilistic Monocular Depth Estimates](https://mlsanity.com/paper/1906.05739/) (2019)

## Built on by (most cited first, 8 of 8)

- [Learning a Depth Covariance Function](https://mlsanity.com/paper/2303.12157/) (2023)
- [DUDES: Deep Uncertainty Distillation using Ensembles for Semantic Segmentation](https://mlsanity.com/paper/2303.09843/) (2023)
- [Efficient Multi-task Uncertainties for Joint Semantic Segmentation and Monocular Depth Estimation](https://mlsanity.com/paper/2402.10580/) (2024)
- [TIC-TAC: A Framework for Improved Covariance Estimation in Deep Heteroscedastic Regression](https://mlsanity.com/paper/2310.18953/) (2023)
- [Multi-Token Enhancing for Vision Representation Learning](https://mlsanity.com/paper/2411.15787/) (2024)
- [Towards Self-Supervised Covariance Estimation in Deep Heteroscedastic Regression](https://mlsanity.com/paper/2502.10587/) (2025)
- [Density Uncertainty Quantification with NeRF-Ensembles: Impact of Data and Scene Constraints](https://mlsanity.com/paper/2312.14664/) (2023)
- [Two-Step MV-DeepONet: Probabilistic Operator Learning for Uncertainty Propagation Driven by Random Input Fields](https://mlsanity.com/paper/2608.09071/) (2026)

## Abstract (arXiv)

This paper proposes using a sparse-structured multivariate Gaussian to provide a closed-form approximator for the output of probabilistic ensemble models used for dense image prediction tasks. This is achieved through a convolutional neural network that predicts the mean and covariance of the distribution, where the inverse covariance is parameterised by a sparsely structured Cholesky matrix. Similarly to distillation approaches, our single network is trained to maximise the probability of samples from pre-trained probabilistic models, in this work we use a fixed ensemble of networks. Once trained, our compact representation can be used to efficiently draw spatially correlated samples from the approximated output distribution. Importantly, this approach captures the uncertainty and structured correlations in the predictions explicitly in a formal distribution, rather than implicitly through sampling alone. This allows direct introspection of the model, enabling visualisation of the learned structure. Moreover, this formulation provides two further benefits: estimation of a sample probability, and the introduction of arbitrary spatial conditioning at test time. We demonstrate the merits of our approach on monocular depth estimation and show that the advantages of our approach are obtained with comparable quantitative performance.