mlsanityData to 7 Oct 2026

Reliable training and estimation of variance networks

arXiv 1906.03260 ↗First posted 4 Jun 2019stat.ML cs.LGcited by 63

Not on the map. Why?

Abstract (arXiv)

We propose and investigate new complementary methodologies for estimating predictive variance networks in regression neural networks. We derive a locally aware mini-batching scheme that result in sparse robust gradients, and show how to make unbiased weight updates to a variance network. Further, we formulate a heuristic for robustly fitting both the mean and variance networks post hoc. Finally, we take inspiration from posterior Gaussian processes and propose a network architecture with similar extrapolation properties to Gaussian processes. The proposed methodologies are complementary, and improve upon baseline methods individually. Experimentally, we investigate the impact on predictive uncertainty on multiple datasets and tasks ranging from regression, active learning and generative modeling. Experiments consistently show significant improvements in predictive uncertainty estimation over state-of-the-art methods across tasks and datasets.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 13; 2023: 16; 2024: 7; 2025: 10; 2026: 9.

63 papers.

Sort:

By area:

  1. 2022On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networkscited by 119
  2. 2022Do Bayesian Neural Networks Need To Be Fully Stochastic?cited by 63
  3. 2021Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantificationcited by 57
  4. 2022A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awarenesscited by 49
  5. 2021Estimating and Evaluating Regression Predictive Uncertainty in Deep Object Detectorscited by 37
  6. 2022Faithful Heteroscedastic Regression with Neural Networkscited by 36
  7. 2023UnCRtainTS: Uncertainty Quantification for Cloud Removal in Optical Satellite Time Seriescited by 28
  8. 2021Pulling back information geometrycited by 23
  9. 2021Calibrated Uncertainty for Molecular Property Prediction using Ensembles of Message Passing Neural Networkscited by 21
  10. 2023Optimal Training of Mean Variance Estimation Neural Networkscited by 17
  11. 2021I Find Your Lack of Uncertainty in Computer Vision Disturbingcited by 16
  12. 2023Uncertainty Quantification via Neural Posterior Principal Componentscited by 16
38 more
  1. 2023Bayesian Metric Learning for Uncertainty Quantification in Image Retrievalcited by 9
  2. 2023A Benchmark on Uncertainty Quantification for Deep Learning Prognosticscited by 9
  3. 2023Distributional Learning of Variational AutoEncoder: Application to Synthetic Data Generationcited by 9
  4. 2023Understanding Pathologies of Deep Heteroskedastic Regressioncited by 9
  5. 2024Counterfactual Explanations via Riemannian Latent Space Traversalcited by 9
  6. 2021Energy-Based Generative Cooperative Saliency Predictioncited by 8
  7. 2024Uncertainty Quantification for Forward and Inverse Problems of PDEs via Latent Global Evolutioncited by 8
  8. 2024Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noisecited by 8
  9. 2025End-to-End Probabilistic Framework for Learning with Hard Constraintscited by 8
  10. 2023Robust Estimation of Causal Heteroscedastic Noise Modelscited by 6
  11. 2024Decoder ensembling for learned latent geometriescited by 6
  12. 2025Probabilistic Machine Learning for Noisy Labels in Earth Observationcited by 6
  13. 2025Uncertainty Quantification for Regression using Proper Scoring Rulescited by 5
  14. 2021Deep Quantile Regression for Uncertainty Estimation in Unsupervised and Supervised Lesion Detectioncited by 4
  15. 2022Estimating Regression Predictive Distributions with Sample Networkscited by 4
  16. 2023TIC-TAC: A Framework for Improved Covariance Estimation in Deep Heteroscedastic Regressioncited by 4
  17. 2021Exploring Uncertainty in Deep Learning for Construction of Prediction Intervalscited by 3
  18. 2022Robust uncertainty estimates with out-of-distribution pseudo-inputs trainingcited by 3
  19. 2022Object Detection as Probabilistic Set Predictioncited by 3
  20. 2022Structured Uncertainty in the Observation Space of Variational Autoencoderscited by 3
  21. 2022Accurate Prediction and Uncertainty Estimation using Decoupled Prediction Interval Networkscited by 2
  22. 2022A view on model misspecification in uncertainty quantificationcited by 2
  23. 2024Optimizing Sharpe Ratio: Risk-Adjusted Decision-Making in Multi-Armed Banditscited by 2
  24. 2024Neural Network Prediction of Strong Lensing Systems with Domain Adaptation and Uncertainty Quantificationcited by 2
  25. 2025Towards Self-Supervised Covariance Estimation in Deep Heteroscedastic Regressioncited by 2
  26. 2022MarkerMap: nonlinear marker selection for single-cell studiescited by 1
  27. 2022Ensemble Multi-Quantiles: Adaptively Flexible Distribution Prediction for Uncertainty Quantificationcited by 1
  28. 2022Improving Uncertainty Quantification of Variance Networks by Tree-Structured Learningcited by 1
  29. 2023Variational Autoencoding of Dental Point Cloudscited by 1
  30. 2023Beta quantile regression for robust estimation of uncertainty in the presence of outlierscited by 1
  31. 2025Reweighting Improves Conditional Risk Boundscited by 1
  32. 2025Cooperative Variance Estimation and Bayesian Neural Networks for Disentangling Aleatoric and Epistemic Uncertaintiescited by 1
  33. 2025Towards Sharper Object Boundaries in Self-Supervised Depth Estimationcited by 1
  34. 2023Integrating Uncertainty into Neural Network-based Speech Enhancementcited by 0
  35. 2023The kernel-balanced equation for deep neural networkscited by 0
  36. 2023Auto-encoding GPS data to reveal individual and collective behaviourcited by 0
  37. 2023Coherent energy and force uncertainty in deep learning force fieldscited by 0
  38. 2023Meta-learning to Calibrate Gaussian Processes with Deep Kernels for Regression Uncertainty Estimationcited by 0

50 most cited of 63 All 63 (JSON)