# Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliability

arXiv: https://arxiv.org/abs/2506.07804 · HTML: https://mlsanity.com/paper/2506.07804/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 50 references that are in the map.
- First posted on arXiv 9 Jun 2025.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Uncertainty Sets for Image Classifiers using Conformal Prediction](https://mlsanity.com/paper/2009.14193/) (2020)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Attacks Which Do Not Kill Training Make Adversarial Learning Stronger](https://mlsanity.com/paper/2002.11242/) (2020)
- [Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methods](https://mlsanity.com/paper/1902.08297/) (2019)
- [Learning Optimal Conformal Classifiers](https://mlsanity.com/paper/2110.09192/) (2021)
- [Robust Validation: Confident Predictions Even When Distributions Shift](https://mlsanity.com/paper/2008.04267/) (2020)
- [Provably Robust Conformal Prediction with Improved Efficiency](https://mlsanity.com/paper/2404.19651/) (2024)
- [Probabilistically robust conformal prediction](https://mlsanity.com/paper/2307.16360/) (2023)
- [Trustworthy Classification through Rank-Based Conformal Prediction Sets](https://mlsanity.com/paper/2407.04407/) (2024)
- [Anomalous Edge Detection in Edge Exchangeable Social Network Models](https://mlsanity.com/paper/2109.12727/) (2021)
- [Robust Yet Efficient Conformal Prediction Sets](https://mlsanity.com/paper/2407.09165/) (2024)
- [Adversarial Training Should Be Cast as a Non-Zero-Sum Game](https://mlsanity.com/paper/2306.11035/) (2023)
- [Conformal Load Prediction with Transductive Graph Autoencoders](https://mlsanity.com/paper/2406.08281/) (2024)
- [Conformal Thresholded Intervals for Efficient Regression](https://mlsanity.com/paper/2407.14495/) (2024)
- [Verifiably Robust Conformal Prediction](https://mlsanity.com/paper/2405.18942/) (2024)
- [Game-Theoretic Defenses for Robust Conformal Prediction Against Adversarial Attacks in Medical Imaging](https://mlsanity.com/paper/2411.04376/) (2024)
- [The Pitfalls and Promise of Conformal Inference Under Adversarial Attacks](https://mlsanity.com/paper/2405.08886/) (2024)
- [Entropy Reweighted Conformal Classification](https://mlsanity.com/paper/2407.17377/) (2024)
- [Conformalized Interval Arithmetic with Symmetric Calibration](https://mlsanity.com/paper/2408.10939/) (2024)

## Built on by (most cited first, 6 of 6)

- [Fast Conformal Prediction using Conditional Interquantile Intervals](https://mlsanity.com/paper/2601.02769/) (2026)
- [A Minimum Variance Path Principle for Accurate and Stable Score-Based Density Ratio Estimation](https://mlsanity.com/paper/2602.00834/) (2026)
- [Towards Disentangled Preference Optimization Dynamics: Suppress the Loser, Preserve the Winner](https://mlsanity.com/paper/2604.18239/) (2026)
- [Cost-Sensitive Conformal Training with Provably Controllable Learning Bounds](https://mlsanity.com/paper/2511.17861/) (2025)
- [Ensuring Calibration Robustness in Split Conformal Prediction Under Adversarial Attacks](https://mlsanity.com/paper/2511.18562/) (2025)
- [Conformal Prediction for Offensive Security](https://mlsanity.com/paper/2609.05165/) (2026)

## Abstract (arXiv)

As deep learning models are increasingly deployed in high-risk applications, robust defenses against adversarial attacks and reliable performance guarantees become paramount. Moreover, accuracy alone does not provide sufficient assurance or reliable uncertainty estimates for these models. This study advances adversarial training by leveraging principles from Conformal Prediction. Specifically, we develop an adversarial attack method, termed OPSA (OPtimal Size Attack), designed to reduce the efficiency of conformal prediction at any significance level by maximizing model uncertainty without requiring coverage guarantees. Correspondingly, we introduce OPSA-AT (Adversarial Training), a defense strategy that integrates OPSA within a novel conformal training paradigm. Experimental evaluations demonstrate that our OPSA attack method induces greater uncertainty compared to baseline approaches for various defenses. Conversely, our OPSA-AT defensive model significantly enhances robustness not only against OPSA but also other adversarial attacks, and maintains reliable prediction. Our findings highlight the effectiveness of this integrated approach for developing trustworthy and resilient deep learning models for safety-critical domains. Our code is available at https://github.com/bjbbbb/Enhancing-Adversarial-Robustness-with-Conformal-Prediction.