# Evaluating Aleatoric Uncertainty via Conditional Generative Models

arXiv: https://arxiv.org/abs/2206.04287 · HTML: https://mlsanity.com/paper/2206.04287/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 55 references that are in the map.
- First posted on arXiv 9 Jun 2022.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Revisiting the Calibration of Modern Neural Networks](https://mlsanity.com/paper/2106.07998/) (2021)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration](https://mlsanity.com/paper/1910.12656/) (2019)
- [Verified Uncertainty Calibration](https://mlsanity.com/paper/1909.10155/) (2019)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [Are GANs Created Equal? A Large-Scale Study](https://mlsanity.com/paper/1711.10337/) (2017)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [MMD GAN: Towards Deeper Understanding of Moment Matching Network](https://mlsanity.com/paper/1705.08584/) (2017)
- [Learning Deep Kernels for Non-Parametric Two-Sample Tests](https://mlsanity.com/paper/2002.09116/) (2020)
- [High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach](https://mlsanity.com/paper/1802.07167/) (2018)
- [Distribution Calibration for Regression](https://mlsanity.com/paper/1905.06023/) (2019)
- [A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings](https://mlsanity.com/paper/2002.03689/) (2020)
- [Maximum Mean Discrepancy Test is Aware of Adversarial Attacks](https://mlsanity.com/paper/2010.11415/) (2020)
- [Calibrated Reliable Regression using Maximum Mean Discrepancy](https://mlsanity.com/paper/2006.10255/) (2020)
- [Fast calibrated additive quantile regression](https://mlsanity.com/paper/1707.03307/) (2017)
- [Conditional Generative Moment-Matching Networks](https://mlsanity.com/paper/1606.04218/) (2016)
- [Gaussian Process Conditional Density Estimation](https://mlsanity.com/paper/1810.12750/) (2018)
- [Conditional Density Estimation Tools in Python and R with Applications to Photometric Redshifts and Likelihood-Free Cosmological Inference](https://mlsanity.com/paper/1908.11523/) (2019)

## Built on by (most cited first, 10 of 10)

- [A Kernel-Based Conditional Two-Sample Test Using Nearest Neighbors (with Applications to Calibration, Regression Curves, and Simulation-Based Inference)](https://mlsanity.com/paper/2407.16550/) (2024)
- [Learning to Forecast Aleatoric and Epistemic Uncertainties over Long Horizon Trajectories](https://mlsanity.com/paper/2302.08669/) (2023)
- [Measuring Differences between Conditional Distributions using Kernel Embeddings](https://mlsanity.com/paper/2605.02260/) (2026)
- [Contextual Scenario Generation for Two-Stage Stochastic Programming](https://mlsanity.com/paper/2502.05349/) (2025)
- [Boosting Statistic Learning with Synthetic Data from Pretrained Large Models](https://mlsanity.com/paper/2505.04992/) (2025)
- [Normalizing Flow to Augmented Posterior: Conditional Density Estimation with Interpretable Dimension Reduction for High Dimensional Data](https://mlsanity.com/paper/2507.04216/) (2025)
- [One-shot Conditional Sampling: MMD meets Nearest Neighbors](https://mlsanity.com/paper/2509.25507/) (2025)
- [Optimal Mixture-of-Experts Model Averaging for Conditional Generative Models](https://mlsanity.com/paper/2607.04360/) (2026)
- [Integrating Contextual Embeddings into Evaluation of Expressive MIDI Piano Performances](https://mlsanity.com/paper/2607.27909/) (2026)
- [Testing Conditional Distribution Equality Using Generative Models](https://mlsanity.com/paper/2610.20447/) (2026)

## Abstract (arXiv)

Aleatoric uncertainty quantification seeks for distributional knowledge of random responses, which is important for reliability analysis and robustness improvement in machine learning applications. Previous research on aleatoric uncertainty estimation mainly targets closed-formed conditional densities or variances, which requires strong restrictions on the data distribution or dimensionality. To overcome these restrictions, we study conditional generative models for aleatoric uncertainty estimation. We introduce two metrics to measure the discrepancy between two conditional distributions that suit these models. Both metrics can be easily and unbiasedly computed via Monte Carlo simulation of the conditional generative models, thus facilitating their evaluation and training. We demonstrate numerically how our metrics provide correct measurements of conditional distributional discrepancies and can be used to train conditional models competitive against existing benchmarks.