# Strength in Numbers: Trading-off Robustness and Computation via Adversarially-Trained Ensembles

arXiv: https://arxiv.org/abs/1811.09300 · HTML: https://mlsanity.com/paper/1811.09300/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2018.

## Built on by (most cited first, 8 of 8)

- [Fixing Data Augmentation to Improve Adversarial Robustness](https://mlsanity.com/paper/2103.01946/) (2021)
- [Data Augmentation Can Improve Robustness](https://mlsanity.com/paper/2111.05328/) (2021)
- [Relating Adversarially Robust Generalization to Flat Minima](https://mlsanity.com/paper/2104.04448/) (2021)
- [How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review](https://mlsanity.com/paper/2107.12045/) (2021)
- [Machine Learning Robustness: A Primer](https://mlsanity.com/paper/2404.00897/) (2024)
- [Drowning in Documents: Consequences of Scaling Reranker Inference](https://mlsanity.com/paper/2411.11767/) (2024)
- [Resisting Adversarial Attacks in Deep Neural Networks using Diverse Decision Boundaries](https://mlsanity.com/paper/2208.08697/) (2022)
- [Discretization-based ensemble model for robust learning in IoT](https://mlsanity.com/paper/2307.08955/) (2023)

## Abstract (arXiv)

While deep learning has led to remarkable results on a number of challenging problems, researchers have discovered a vulnerability of neural networks in adversarial settings, where small but carefully chosen perturbations to the input can make the models produce extremely inaccurate outputs. This makes these models particularly unsuitable for safety-critical application domains (e.g. self-driving cars) where robustness is extremely important. Recent work has shown that augmenting training with adversarially generated data provides some degree of robustness against test-time attacks. In this paper we investigate how this approach scales as we increase the computational budget given to the defender. We show that increasing the number of parameters in adversarially-trained models increases their robustness, and in particular that ensembling smaller models while adversarially training the entire ensemble as a single model is a more efficient way of spending said budget than simply using a larger single model. Crucially, we show that it is the adversarial training of the ensemble, rather than the ensembling of adversarially trained models, which provides robustness.