# DVERGE: Diversifying Vulnerabilities for Enhanced Robust Generation of Ensembles

arXiv: https://arxiv.org/abs/2009.14720 · HTML: https://mlsanity.com/paper/2009.14720/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2020.

## Built on by (most cited first, 30 of 73)

- [Recent Advances in Adversarial Training for Adversarial Robustness](https://mlsanity.com/paper/2102.01356/) (2021)
- [Cross-Entropy Loss Functions: Theoretical Analysis and Applications](https://mlsanity.com/paper/2304.07288/) (2023)
- [3D Common Corruptions and Data Augmentation](https://mlsanity.com/paper/2203.01441/) (2022)
- [Random Noise Defense Against Query-Based Black-Box Attacks](https://mlsanity.com/paper/2104.11470/) (2021)
- [An Adaptive Model Ensemble Adversarial Attack for Boosting Adversarial Transferability](https://mlsanity.com/paper/2308.02897/) (2023)
- [On the Certified Robustness for Ensemble Models and Beyond](https://mlsanity.com/paper/2107.10873/) (2021)
- [TRS: Transferability Reduced Ensemble via Encouraging Gradient Diversity and Model Smoothness](https://mlsanity.com/paper/2104.00671/) (2021)
- [GradDiv: Adversarial Robustness of Randomized Neural Networks via Gradient Diversity Regularization](https://mlsanity.com/paper/2107.02425/) (2021)
- [Knowledge Enhanced Machine Learning Pipeline against Diverse Adversarial Attacks](https://mlsanity.com/paper/2106.06235/) (2021)
- [Robustness via Cross-Domain Ensembles](https://mlsanity.com/paper/2103.10919/) (2021)
- [Self-Ensemble Protection: Training Checkpoints Are Good Data Protectors](https://mlsanity.com/paper/2211.12005/) (2022)
- [Introducing Competition to Boost the Transferability of Targeted Adversarial Examples through Clean Feature Mixup](https://mlsanity.com/paper/2305.14846/) (2023)
- [Orthogonal Ensemble Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/2105.10827/) (2021)
- [Two Coupled Rejection Metrics Can Tell Adversarial Examples Apart](https://mlsanity.com/paper/2105.14785/) (2021)
- [Understanding and Improving Ensemble Adversarial Defense](https://mlsanity.com/paper/2310.18477/) (2023)
- [Defenses in Adversarial Machine Learning: A Survey](https://mlsanity.com/paper/2312.08890/) (2023)
- [DI-AA: An Interpretable White-box Attack for Fooling Deep Neural Networks](https://mlsanity.com/paper/2110.07305/) (2021)
- [Collaboration! Towards Robust Neural Methods for Routing Problems](https://mlsanity.com/paper/2410.04968/) (2024)
- [NoiLIn: Improving Adversarial Training and Correcting Stereotype of Noisy Labels](https://mlsanity.com/paper/2105.14676/) (2021)
- [Domain Invariant Adversarial Learning](https://mlsanity.com/paper/2104.00322/) (2021)
- [Towards Comprehensive Monocular Depth Estimation: Multiple Heads Are Better Than One](https://mlsanity.com/paper/2111.08313/) (2021)
- [Adaptive Perturbation for Adversarial Attack](https://mlsanity.com/paper/2111.13841/) (2021)
- [MORA: Improving Ensemble Robustness Evaluation with Model-Reweighing Attack](https://mlsanity.com/paper/2211.08008/) (2022)
- [PubDef: Defending Against Transfer Attacks From Public Models](https://mlsanity.com/paper/2310.17645/) (2023)
- [Mixture of Robust Experts (MoRE):A Robust Denoising Method towards multiple perturbations](https://mlsanity.com/paper/2104.10586/) (2021)
- [Disrupting Adversarial Transferability in Deep Neural Networks](https://mlsanity.com/paper/2108.12492/) (2021)
- [Push Stricter to Decide Better: A Class-Conditional Feature Adaptive Framework for Improving Adversarial Robustness](https://mlsanity.com/paper/2112.00323/) (2021)
- [Adversarial for Good? How the Adversarial ML Community's Values Impede Socially Beneficial Uses of Attacks](https://mlsanity.com/paper/2107.10302/) (2021)
- [Adversarial Vulnerability of Randomized Ensembles](https://mlsanity.com/paper/2206.06737/) (2022)
- [Investigating Catastrophic Overfitting in Fast Adversarial Training: A Self-fitting Perspective](https://mlsanity.com/paper/2302.11963/) (2023)

## Abstract (arXiv)

Recent research finds CNN models for image classification demonstrate overlapped adversarial vulnerabilities: adversarial attacks can mislead CNN models with small perturbations, which can effectively transfer between different models trained on the same dataset. Adversarial training, as a general robustness improvement technique, eliminates the vulnerability in a single model by forcing it to learn robust features. The process is hard, often requires models with large capacity, and suffers from significant loss on clean data accuracy. Alternatively, ensemble methods are proposed to induce sub-models with diverse outputs against a transfer adversarial example, making the ensemble robust against transfer attacks even if each sub-model is individually non-robust. Only small clean accuracy drop is observed in the process. However, previous ensemble training methods are not efficacious in inducing such diversity and thus ineffective on reaching robust ensemble. We propose DVERGE, which isolates the adversarial vulnerability in each sub-model by distilling non-robust features, and diversifies the adversarial vulnerability to induce diverse outputs against a transfer attack. The novel diversity metric and training procedure enables DVERGE to achieve higher robustness against transfer attacks comparing to previous ensemble methods, and enables the improved robustness when more sub-models are added to the ensemble. The code of this work is available at https://github.com/zjysteven/DVERGE