# Diverse Ensembles Improve Calibration

arXiv: https://arxiv.org/abs/2007.04206 · HTML: https://mlsanity.com/paper/2007.04206/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jul 2020.

## Built on by (most cited first, 23 of 23)

- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions](https://mlsanity.com/paper/2308.11483/) (2023)
- [Agree to Disagree: Diversity through Disagreement for Better Transferability](https://mlsanity.com/paper/2202.04414/) (2022)
- [DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling](https://mlsanity.com/paper/2105.12441/) (2021)
- [DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation](https://mlsanity.com/paper/2101.05544/) (2021)
- [Knowing More About Questions Can Help: Improving Calibration in Question Answering](https://mlsanity.com/paper/2106.01494/) (2021)
- [Weighted Ensemble Models Are Strong Continual Learners](https://mlsanity.com/paper/2312.08977/) (2023)
- [Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift](https://mlsanity.com/paper/2207.08977/) (2022)
- [Orthogonal Ensemble Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/2105.10827/) (2021)
- [Query-Level Uncertainty in Large Language Models](https://mlsanity.com/paper/2506.09669/) (2025)
- [Maximum Entropy on Erroneous Predictions (MEEP): Improving model calibration for medical image segmentation](https://mlsanity.com/paper/2112.12218/) (2021)
- [Combining Diverse Feature Priors](https://mlsanity.com/paper/2110.08220/) (2021)
- [Should Ensemble Members Be Calibrated?](https://mlsanity.com/paper/2101.05397/) (2021)
- [FAIR-Ensemble: When Fairness Naturally Emerges From Deep Ensembling](https://mlsanity.com/paper/2303.00586/) (2023)
- [Improving robustness and calibration in ensembles with diversity regularization](https://mlsanity.com/paper/2201.10908/) (2022)
- [DIVE: Subgraph Disagreement for Graph Out-of-Distribution Generalization](https://mlsanity.com/paper/2408.04400/) (2024)
- [Combining Different V1 Brain Model Variants to Improve Robustness to Image Corruptions in CNNs](https://mlsanity.com/paper/2110.10645/) (2021)
- [Bayesian posterior approximation with stochastic ensembles](https://mlsanity.com/paper/2212.08123/) (2022)
- [Multi-Symmetry Ensembles: Improving Diversity and Generalization via Opposing Symmetries](https://mlsanity.com/paper/2303.02484/) (2023)
- [Diversifying Deep Ensembles: A Saliency Map Approach for Enhanced OOD Detection, Calibration, and Accuracy](https://mlsanity.com/paper/2305.11616/) (2023)
- [Parameter-Efficient Construction of the Rashomon Slice for Concept Bottleneck Models](https://mlsanity.com/paper/2511.19636/) (2025)
- [Source-free Domain Adaptation Requires Penalized Diversity](https://mlsanity.com/paper/2304.02798/) (2023)
- [WWAggr: A Window Wasserstein-based Aggregation for Ensemble Change Point Detection](https://mlsanity.com/paper/2506.08066/) (2025)

## Abstract (arXiv)

Modern deep neural networks can produce badly calibrated predictions, especially when train and test distributions are mismatched. Training an ensemble of models and averaging their predictions can help alleviate these issues. We propose a simple technique to improve calibration, using a different data augmentation for each ensemble member. We additionally use the idea of `mixing' un-augmented and augmented inputs to improve calibration when test and training distributions are the same. These simple techniques improve calibration and accuracy over strong baselines on the CIFAR10 and CIFAR100 benchmarks, and out-of-domain data from their corrupted versions.