mlsanityData to 7 Oct 2026

BulletTrain: Accelerating Robust Neural Network Training via Boundary Example Mining

arXiv 2109.14707 ↗First posted 29 Sep 2021cs.LG cs.AI cs.CRcited by 8

Not on the map. Why?

Abstract (arXiv)

Neural network robustness has become a central topic in machine learning in recent years. Most training algorithms that improve the model's robustness to adversarial and common corruptions also introduce a large computational overhead, requiring as many as ten times the number of forward and backward passes in order to converge. To combat this inefficiency, we propose BulletTrain - a boundary example mining technique to drastically reduce the computational cost of robust training. Our key observation is that only a small fraction of examples are beneficial for improving robustness. BulletTrain dynamically predicts these important examples and optimizes robust training algorithms to focus on the important examples. We apply our technique to several existing robust training algorithms and achieve a 2.1× speed-up for TRADES and MART on CIFAR-10 and a 1.7× speed-up for AugMix on CIFAR-10-C and CIFAR-100-C without any reduction in clean and robust accuracy.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 22 of its 28 references that are in the map. First posted on arXiv 29 Sep 2021.

Builds on

22 of 28 references.

References · 22

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  3. 2016Wide Residual Networkscited by 3,057
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  5. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  6. 2016Concrete Problems in AI Safetycited by 1,582
  7. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  8. 2015Prioritized Experience Replaycited by 1,070
  9. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  10. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  11. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  12. 2019Adversarial Training for Free!cited by 638
10 more
  1. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
  2. 2020Hard Negative Mixing for Contrastive Learningcited by 399
  3. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 323
  4. 2017Sampling Matters in Deep Embedding Learningcited by 285
  5. 2020Understanding and Improving Fast Adversarial Trainingcited by 213
  6. 2019You Only Propagate Once: Accelerating Adversarial Training via Maximal Principlecited by 181
  7. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 171
  8. 2017Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samplescited by 147
  9. 2015Online Batch Selection for Faster Training of Neural Networkscited by 121
  10. 2020Efficient Robust Training via Backward Smoothingcited by 29

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 1; 2024: 2; 2025: 3.

8 papers.

  1. 2023Less is More: Data Pruning for Faster Adversarial Trainingcited by 13
  2. 2024SAGMAN: Stability Analysis of Graph Neural Networks on the Manifoldscited by 4
  3. 2021Super-Efficient Super Resolution for Fast Adversarial Defense at the Edgecited by 2
  4. 2024Large-Scale Dataset Pruning in Adversarial Training through Data Importance Extrapolationcited by 2
  5. 2025Effective Data Pruning through Score Extrapolationcited by 2
  6. 2025Robustness Feature Adapter for Efficient Adversarial Trainingcited by 2
  7. 2022A Fast and Efficient Conditional Learning for Tunable Trade-Off between Accuracy and Robustnesscited by 1
  8. 2025Efficient Semi-Supervised Adversarial Training via Latent Clustering-Based Data Reductioncited by 0