# Beware the Black-Box: on the Robustness of Recent Defenses to Adversarial Examples

arXiv: https://arxiv.org/abs/2006.10876 · HTML: https://mlsanity.com/paper/2006.10876/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2020.

## Built on by (most cited first, 11 of 11)

- [On the Robustness of Vision Transformers to Adversarial Examples](https://mlsanity.com/paper/2104.02610/) (2021)
- [Back in Black: A Comparative Evaluation of Recent State-Of-The-Art Black-Box Attacks](https://mlsanity.com/paper/2109.15031/) (2021)
- [Attacking the Spike: On the Transferability and Security of Spiking Neural Networks to Adversarial Examples](https://mlsanity.com/paper/2209.03358/) (2022)
- [Game Theoretic Mixed Experts for Combinational Adversarial Machine Learning](https://mlsanity.com/paper/2211.14669/) (2022)
- [Certifying Adapters: Enabling and Enhancing the Certification of Classifier Adversarial Robustness](https://mlsanity.com/paper/2405.16036/) (2024)
- [RobustBlack: Challenging Black-Box Adversarial Attacks on State-of-the-Art Defenses](https://mlsanity.com/paper/2412.20987/) (2024)
- [Simple black-box universal adversarial attacks on medical image classification based on deep neural networks](https://mlsanity.com/paper/2108.04979/) (2021)
- [Busting the Paper Ballot: Voting Meets Adversarial Machine Learning](https://mlsanity.com/paper/2506.14582/) (2025)
- [Multi-task Adversarial Attacks against Black-box Model with Few-shot Queries](https://mlsanity.com/paper/2508.10039/) (2025)
- [On the Robustness of Distributed Machine Learning against Transfer Attacks](https://mlsanity.com/paper/2412.14080/) (2024)
- [Compression as an Adversarial Amplifier Through Decision Space Reduction](https://mlsanity.com/paper/2604.06954/) (2026)

## Abstract (arXiv)

Many defenses have recently been proposed at venues like NIPS, ICML, ICLR and CVPR. These defenses are mainly focused on mitigating white-box attacks. They do not properly examine black-box attacks. In this paper, we expand upon the analysis of these defenses to include adaptive black-box adversaries. Our evaluation is done on nine defenses including Barrage of Random Transforms, ComDefend, Ensemble Diversity, Feature Distillation, The Odds are Odd, Error Correcting Codes, Distribution Classifier Defense, K-Winner Take All and Buffer Zones. Our investigation is done using two black-box adversarial models and six widely studied adversarial attacks for CIFAR-10 and Fashion-MNIST datasets. Our analyses show most recent defenses (7 out of 9) provide only marginal improvements in security (<25%), as compared to undefended networks. For every defense, we also show the relationship between the amount of data the adversary has at their disposal, and the effectiveness of adaptive black-box attacks. Overall, our results paint a clear picture: defenses need both thorough white-box and black-box analyses to be considered secure. We provide this large scale study and analyses to motivate the field to move towards the development of more robust black-box defenses.