# Test-time augmentation improves efficiency in conformal prediction

arXiv: https://arxiv.org/abs/2505.22764 · HTML: https://mlsanity.com/paper/2505.22764/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 47 references that are in the map.
- First posted on arXiv 28 May 2025.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [AutoAugment: Learning Augmentation Policies from Data](https://mlsanity.com/paper/1805.09501/) (2018)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [MEMO: Test Time Robustness via Adaptation and Augmentation](https://mlsanity.com/paper/2110.09506/) (2021)
- [Benchmarking Representation Learning for Natural World Image Collections](https://mlsanity.com/paper/2103.16483/) (2021)
- [Better Aggregation in Test-Time Augmentation](https://mlsanity.com/paper/2011.11156/) (2020)
- [Learning by Transduction](https://mlsanity.com/paper/1301.7375/) (2013)
- [Class-Conditional Conformal Prediction with Many Classes](https://mlsanity.com/paper/2306.09335/) (2023)
- [Training Uncertainty-Aware Classifiers with Conformalized Deep Learning](https://mlsanity.com/paper/2205.05878/) (2022)
- [Greedy Policy Search: A Simple Baseline for Learnable Test-Time Augmentation](https://mlsanity.com/paper/2002.09103/) (2020)
- [Learning Loss for Test-Time Augmentation](https://mlsanity.com/paper/2010.11422/) (2020)
- [Exchangeability, Conformal Prediction, and Rank Tests](https://mlsanity.com/paper/2005.06095/) (2020)
- [Conformal Prediction Sets Improve Human Decision Making](https://mlsanity.com/paper/2401.13744/) (2024)
- [On the Utility of Prediction Sets in Human-AI Teams](https://mlsanity.com/paper/2205.01411/) (2022)
- [Optimized conformal classification using gradient descent approximation](https://mlsanity.com/paper/2105.11255/) (2021)
- [Enhancing Adversarial Robustness via Test-time Transformation Ensembling](https://mlsanity.com/paper/2107.14110/) (2021)
- [Ensemble Conformalized Quantile Regression for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2202.08756/) (2022)
- [Evaluating the Utility of Conformal Prediction Sets for AI-Advised Image Labeling](https://mlsanity.com/paper/2401.08876/) (2024)
- [Improved Text Classification via Test-Time Augmentation](https://mlsanity.com/paper/2206.13607/) (2022)
- [Approaching Test Time Augmentation in the Context of Uncertainty Calibration for Deep Neural Networks](https://mlsanity.com/paper/2304.05104/) (2023)
- [Simple Post-Training Robustness Using Test Time Augmentations and Random Forest](https://mlsanity.com/paper/2109.08191/) (2021)
- [Dynamic Test-Time Augmentation via Differentiable Functions](https://mlsanity.com/paper/2212.04681/) (2022)
- [Adaptive Test-Time Augmentation for Low-Power CPU](https://mlsanity.com/paper/2105.06183/) (2021)

## Built on by (most cited first, 3 of 3)

- [Look Again Before You Abstain:Budgeted Conformal Evidence Acquisition for Reliable Vision-Language Model](https://mlsanity.com/paper/2606.16667/) (2026)
- [Fast Preemptive Robustification: High-Frequency Response Anti-Aligns Shared Vulnerability](https://mlsanity.com/paper/2603.25244/) (2026)
- [StableTTA: Improving Vision Model Performance by Training-free Test-Time Adaptation Methods](https://mlsanity.com/paper/2604.04552/) (2026)

## Abstract (arXiv)

A conformal classifier produces a set of predicted classes and provides a probabilistic guarantee that the set includes the true class. Unfortunately, it is often the case that conformal classifiers produce uninformatively large sets. In this work, we show that test-time augmentation (TTA)--a technique that introduces inductive biases during inference--reduces the size of the sets produced by conformal classifiers. Our approach is flexible, computationally efficient, and effective. It can be combined with any conformal score, requires no model retraining, and reduces prediction set sizes by 10%-14% on average. We conduct an evaluation of the approach spanning three datasets, three models, two established conformal scoring methods, different guarantee strengths, and several distribution shifts to show when and why test-time augmentation is a useful addition to the conformal pipeline.