# Learning Calibrated Uncertainties for Domain Shift: A Distributionally Robust Learning Approach

arXiv: https://arxiv.org/abs/2010.05784 · HTML: https://mlsanity.com/paper/2010.05784/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2020.

## Built on by (most cited first, 6 of 6)

- [Uncertainty Quantification for Machine Learning in Healthcare: A Survey](https://mlsanity.com/paper/2505.02874/) (2025)
- [Density Ratio Estimation with Conditional Probability Paths](https://mlsanity.com/paper/2502.02300/) (2025)
- [Towards Calibrated Deep Clustering Network](https://mlsanity.com/paper/2403.02998/) (2024)
- [A Minimum Variance Path Principle for Accurate and Stable Score-Based Density Ratio Estimation](https://mlsanity.com/paper/2602.00834/) (2026)
- [Ensemble-Based Dirichlet Modeling for Predictive Uncertainty and Selective Classification](https://mlsanity.com/paper/2604.06032/) (2026)
- [Expectation Consistency Loss: Rethink Confidence Calibration under Covariate Shift](https://mlsanity.com/paper/2605.21552/) (2026)

## Abstract (arXiv)

We propose a framework for learning calibrated uncertainties under domain shifts, where the source (training) distribution differs from the target (test) distribution. We detect such domain shifts via a differentiable density ratio estimator and train it together with the task network, composing an adjusted softmax predictive form concerning domain shift. In particular, the density ratio estimation reflects the closeness of a target (test) sample to the source (training) distribution. We employ it to adjust the uncertainty of prediction in the task network. This idea of using the density ratio is based on the distributionally robust learning (DRL) framework, which accounts for the domain shift by adversarial risk minimization. We show that our proposed method generates calibrated uncertainties that benefit downstream tasks, such as unsupervised domain adaptation (UDA) and semi-supervised learning (SSL). On these tasks, methods like self-training and FixMatch use uncertainties to select confident pseudo-labels for re-training. Our experiments show that the introduction of DRL leads to significant improvements in cross-domain performance. We also show that the estimated density ratios align with human selection frequencies, suggesting a positive correlation with a proxy of human perceived uncertainties.