# D2R: dual regularization loss with collaborative adversarial generation for model robustness

arXiv: https://arxiv.org/abs/2506.07056 · HTML: https://mlsanity.com/paper/2506.07056/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 8 Jun 2025.

## Builds on (most cited first, 17 of 17)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017)
- [Overfitting in adversarially robust deep learning](https://mlsanity.com/paper/2002.11569/) (2020)
- [Adversarially Robust Generalization Requires More Data](https://mlsanity.com/paper/1804.11285/) (2018)
- [Attacks Which Do Not Kill Training Make Adversarial Learning Stronger](https://mlsanity.com/paper/2002.11242/) (2020)
- [Geometry-aware Instance-reweighted Adversarial Training](https://mlsanity.com/paper/2010.01736/) (2020)
- [Adversarially Robust Distillation](https://mlsanity.com/paper/1905.09747/) (2019)
- [LAS-AT: Adversarial Training with Learnable Attack Strategy](https://mlsanity.com/paper/2203.06616/) (2022)
- [Learnable Boundary Guided Adversarial Training](https://mlsanity.com/paper/2011.11164/) (2020)
- [Randomized Adversarial Training via Taylor Expansion](https://mlsanity.com/paper/2303.10653/) (2023)
- [SAT: Improving Adversarial Training via Curriculum-Based Loss Smoothing](https://mlsanity.com/paper/2003.09347/) (2020)
- [Mitigating Accuracy-Robustness Trade-off via Balanced Multi-Teacher Adversarial Distillation](https://mlsanity.com/paper/2306.16170/) (2023)
- [Dynamic Label Adversarial Training for Deep Learning Robustness Against Adversarial Attacks](https://mlsanity.com/paper/2408.13102/) (2024)

## Built on by (most cited first, 1 of 1)

- [AdaGAT: Adaptive Guidance Adversarial Training for the Robustness of Deep Neural Networks](https://mlsanity.com/paper/2508.17265/) (2025)

## Abstract (arXiv)

The robustness of Deep Neural Network models is crucial for defending models against adversarial attacks. Recent defense methods have employed collaborative learning frameworks to enhance model robustness. Two key limitations of existing methods are (i) insufficient guidance of the target model via loss functions and (ii) non-collaborative adversarial generation. We, therefore, propose a dual regularization loss (D2R Loss) method and a collaborative adversarial generation (CAG) strategy for adversarial training. D2R loss includes two optimization steps. The adversarial distribution and clean distribution optimizations enhance the target model's robustness by leveraging the strengths of different loss functions obtained via a suitable function space exploration to focus more precisely on the target model's distribution. CAG generates adversarial samples using a gradient-based collaboration between guidance and target models. We conducted extensive experiments on three benchmark databases, including CIFAR-10, CIFAR-100, Tiny ImageNet, and two popular target models, WideResNet34-10 and PreActResNet18. Our results show that D2R loss with CAG produces highly robust models.