# Where is the Bottleneck of Adversarial Learning with Unlabeled Data?

arXiv: https://arxiv.org/abs/1911.08696 · HTML: https://mlsanity.com/paper/1911.08696/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2019.

## Built on by (most cited first, 3 of 3)

- [Decoupled Adversarial Contrastive Learning for Self-supervised Adversarial Robustness](https://mlsanity.com/paper/2207.10899/) (2022)
- [Understanding the Interaction of Adversarial Training with Noisy Labels](https://mlsanity.com/paper/2102.03482/) (2021)
- [Towards Understanding Dual BN In Hybrid Adversarial Training](https://mlsanity.com/paper/2403.19150/) (2024)

## Abstract (arXiv)

Deep neural networks (DNNs) are incredibly brittle due to adversarial examples. To robustify DNNs, adversarial training was proposed, which requires large-scale but well-labeled data. However, it is quite expensive to annotate large-scale data well. To compensate for this shortage, several seminal works are utilizing large-scale unlabeled data. In this paper, we observe that seminal works do not perform well, since the quality of pseudo labels on unlabeled data is quite poor, especially when the amount of unlabeled data is significantly larger than that of labeled data. We believe that the quality of pseudo labels is the bottleneck of adversarial learning with unlabeled data. To tackle this bottleneck, we leverage deep co-training, which trains two deep networks and encourages two networks diverged by exploiting peer's adversarial examples. Based on deep co-training, we propose robust co-training (RCT) for adversarial learning with unlabeled data. We conduct comprehensive experiments on CIFAR-10 and SVHN datasets. Empirical results demonstrate that our RCT can significantly outperform baselines (e.g., robust self-training (RST)) in both standard test accuracy and robust test accuracy w.r.t. different datasets, different network structures, and different types of adversarial training.