mlsanityData to 14 Oct 2026

Test-time augmentation improves efficiency in conformal prediction

arXiv 2505.22764 ↗First posted 28 May 2025cs.LG cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

A conformal classifier produces a set of predicted classes and provides a probabilistic guarantee that the set includes the true class. Unfortunately, it is often the case that conformal classifiers produce uninformatively large sets. In this work, we show that test-time augmentation (TTA)--a technique that introduces inductive biases during inference--reduces the size of the sets produced by conformal classifiers. Our approach is flexible, computationally efficient, and effective. It can be combined with any conformal score, requires no model retraining, and reduces prediction set sizes by 10%-14% on average. We conduct an evaluation of the approach spanning three datasets, three models, two established conformal scoring methods, different guarantee strengths, and several distribution shifts to show when and why test-time augmentation is a useful addition to the conformal pipeline.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 28 of its 47 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

28 of 47 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2013Intriguing properties of neural networkscited by 5,155
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  5. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  6. 2018AutoAugment: Learning Augmentation Policies from Datacited by 730
  7. 2019Conformal Prediction Under Covariate Shiftcited by 519
  8. 2020Classification with Valid and Adaptive Coveragecited by 396
  9. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 377
  10. 2021Benchmarking Representation Learning for Natural World Image Collectionscited by 168
  11. 2020Better Aggregation in Test-Time Augmentationcited by 122
  12. 2013Learning by Transductioncited by 121
16 more
  1. 2023Class-Conditional Conformal Prediction with Many Classescited by 109
  2. 2022Training Uncertainty-Aware Classifiers with Conformalized Deep Learningcited by 64
  3. 2020Greedy Policy Search: A Simple Baseline for Learnable Test-Time Augmentationcited by 61
  4. 2020Learning Loss for Test-Time Augmentationcited by 56
  5. 2020Exchangeability, Conformal Prediction, and Rank Testscited by 41
  6. 2024Conformal Prediction Sets Improve Human Decision Makingcited by 35
  7. 2022On the Utility of Prediction Sets in Human-AI Teamscited by 34
  8. 2021Optimized conformal classification using gradient descent approximationcited by 28
  9. 2021Enhancing Adversarial Robustness via Test-time Transformation Ensemblingcited by 27
  10. 2022Ensemble Conformalized Quantile Regression for Probabilistic Time Series Forecastingcited by 27
  11. 2024Evaluating the Utility of Conformal Prediction Sets for AI-Advised Image Labelingcited by 12
  12. 2022Improved Text Classification via Test-Time Augmentationcited by 7
  13. 2023Approaching Test Time Augmentation in the Context of Uncertainty Calibration for Deep Neural Networkscited by 7
  14. 2021Simple Post-Training Robustness Using Test Time Augmentations and Random Forestcited by 4
  15. 2022Dynamic Test-Time Augmentation via Differentiable Functionscited by 4
  16. 2021Adaptive Test-Time Augmentation for Low-Power CPUcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Look Again Before You Abstain:Budgeted Conformal Evidence Acquisition for Reliable Vision-Language Modelcited by 3
  2. 2026Fast Preemptive Robustification: High-Frequency Response Anti-Aligns Shared Vulnerabilitycited by 0
  3. 2026StableTTA: Improving Vision Model Performance by Training-free Test-Time Adaptation Methodscited by 0