Efficient Semi-Supervised Adversarial Training via Latent Clustering-Based Data Reduction
Not on the map. Why?
Abstract (arXiv)
Learning robust models under adversarial settings is widely recognized as requiring a considerably large number of training samples. Recent work proposes semi-supervised adversarial training (SSAT), which utilizes external unlabeled or synthetically generated data and is currently the state of the art. However, SSAT requires substantial extra data to attain high robustness, resulting in prolonged training time and increased memory usage. In this paper, we propose data reduction strategies to improve the efficiency of SSAT by optimizing the amount of additional data incorporated. Specifically, we design novel latent clustering-based techniques to select or generate a small, critical subset of data samples near the model's decision boundary. While focusing on boundary-adjacent points, our methods maintain a balanced ratio between boundary and non-boundary data points, thereby avoiding overfitting. Comprehensive experiments across image benchmarks demonstrate that our methods can effectively reduce SSAT's data requirements and computational costs while preserving its strong robustness advantages. In particular, our latent-space selection scheme based on k-means clustering and our guided diffusion-based approach with LCG-KM are the most effective, achieving nearly identical robust accuracies with 5 times to 10 times less unlabeled data. When compared to full SSAT trained to convergence, our methods reduce total runtime by approximately 3 times to 4 times due to strategic prioritization of unlabeled data.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 41 of its 57 references that are in the map. First posted on arXiv 15 Jan 2025.
Builds on
41 of 57 references.
References · 41
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2013Intriguing properties of neural networkscited by 5,155
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2016Wide Residual Networkscited by 3,070
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
29 more
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
- 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
- 2020Fast is better than free: Revisiting adversarial trainingcited by 727
- 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 678
- 2020RobustBench: a standardized adversarial robustness benchmarkcited by 661
- 2019Adversarial Training for Free!cited by 643
- 2020Dataset Condensation with Gradient Matchingcited by 516
- 2020Overfitting in adversarially robust deep learningcited by 506
- 2019Unlabeled Data Improves Adversarial Robustnesscited by 416
- 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 414
- 2018Adversarially Robust Generalization Requires More Datacited by 318
- 2021Improving Robustness using Generated Datacited by 234
- 2018Dataset Distillationcited by 230
- 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 208
- 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
- 2019Are Labels Required for Improving Adversarial Robustness?cited by 163
- 2021Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?cited by 113
- 2019Soft-Label Dataset Distillation and Text Dataset Distillationcited by 107
- 2019Adversarially Robust Generalization Just Requires More Unlabeled Datacited by 76
- 2019Learning From Less Data: A Unified Data Subset Selection and Active Learning Framework for Computer Visioncited by 64
- 2019Robustness to Adversarial Perturbations in Learning from Incomplete Datacited by 49
- 2020COVIDGR dataset and COVID-SDNet methodology for predicting COVID-19 based on Chest X-Ray imagescited by 32
- 2020A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via Adversarial Fine-tuningcited by 30
- 2020Efficient Robust Training via Backward Smoothingcited by 29
- 2021Revisiting Dimensionality Reduction Techniques for Visual Cluster Analysis: An Empirical Studycited by 27
- 2022In Defense of Core-set: A Density-aware Core-set Selection for Active Learningcited by 27
- 2018Principal Component Analysis: A Natural Approach to Data Explorationcited by 9
- 2022Adversarial Coreset Selection for Efficient Robust Trainingcited by 9
- 2021BulletTrain: Accelerating Robust Neural Network Training via Boundary Example Miningcited by 8
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Enhancing Adversarial Robustness in Low-Label Regime via Adaptively Weighted Regularization and Knowledge Distillation
- 2023ClusterDDPM: An EM clustering framework with Denoising Diffusion Probabilistic Models
- 2024Rethinking Invariance Regularization in Adversarial Training to Improve Robustness-Accuracy Trade-off
- 2026Efficient LLM Adversarial Training via Low-Rank Defense and Circuit-Guided Surrogates
- 2024Explaining the role of Intrinsic Dimensionality in Adversarial Training
- 2024Exploring Adversarial Robustness of Deep State Space Models
- 2022(Certified!!) Adversarial Robustness for Free!
- 2022Towards Alternative Techniques for Improving Adversarial Robustness: Analysis of Adversarial Training at a Spectrum of Perturbations