# Variational Variance: Simple, Reliable, Calibrated Heteroscedastic Noise Variance Parameterization

arXiv: https://arxiv.org/abs/2006.04910 · HTML: https://mlsanity.com/paper/2006.04910/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2020.

## Built on by (most cited first, 5 of 5)

- [On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks](https://mlsanity.com/paper/2203.09168/) (2022)
- [Faithful Heteroscedastic Regression with Neural Networks](https://mlsanity.com/paper/2212.09184/) (2022)
- [Deep Quantile Regression for Uncertainty Estimation in Unsupervised and Supervised Lesion Detection](https://mlsanity.com/paper/2109.09374/) (2021)
- [Robust uncertainty estimates with out-of-distribution pseudo-inputs training](https://mlsanity.com/paper/2201.05890/) (2022)
- [Structured Uncertainty in the Observation Space of Variational Autoencoders](https://mlsanity.com/paper/2205.12533/) (2022)

## Abstract (arXiv)

Brittle optimization has been observed to adversely impact model likelihoods for regression and VAEs when simultaneously fitting neural network mappings from a (random) variable onto the mean and variance of a dependent Gaussian variable. Previous works have bolstered optimization and improved likelihoods, but fail other basic posterior predictive checks (PPCs). Under the PPC framework, we propose critiques to test predictive mean and variance calibration and the predictive distribution's ability to generate sensible data. We find that our attractively simple solution, to treat heteroscedastic variance variationally, sufficiently regularizes variance to pass these PPCs. We consider a diverse gamut of existing and novel priors and find our methods preserve or outperform existing model likelihoods while significantly improving parameter calibration and sample quality for regression and VAEs.