mlsanityData to 7 Oct 2026

Adversarial Training is Not Ready for Robot Learning

arXiv 2103.08187 ↗First posted 15 Mar 2021cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

Adversarial training is an effective method to train deep learning models that are resilient to norm-bounded perturbations, with the cost of nominal performance drop. While adversarial training appears to enhance the robustness and safety of a deep model deployed in open-world decision-critical applications, counterintuitively, it induces undesired behaviors in robot learning settings. In this paper, we show theoretically and experimentally that neural controllers obtained via adversarial training are subjected to three types of defects, namely transient, systematic, and conditional errors. We first generalize adversarial training to a safety-domain optimization scheme allowing for more generic specifications. We then prove that such a learning process tends to cause certain error profiles. We support our theoretical results by a thorough experimental safety analysis in a robot-learning task. Our results suggest that adversarial training is not yet ready for robot learning.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 50 of its 68 references that are in the map. First posted on arXiv 15 Mar 2021.

Builds on

50 of 68 references.

References · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  7. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  8. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  9. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  10. 2016Adversarial Machine Learning at Scalecited by 934
  11. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  12. 2018Robustness May Be at Odds with Accuracycited by 873
38 more
  1. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  2. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
  3. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  4. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 462
  5. 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 328
  6. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 322
  7. 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 315
  8. 2018Feature Denoising for Improving Adversarial Robustnesscited by 312
  9. 2018Certified Defenses against Adversarial Examplescited by 305
  10. 2018Towards Fast Computation of Certified Robustness for ReLU Networkscited by 254
  11. 2019Causality for Machine Learningcited by 242
  12. 2018Adversarial Risk and the Dangers of Evaluating Against Weak Attackscited by 221
  13. 2018Disentangling Adversarial Robustness and Generalizationcited by 171
  14. 2020Liquid Time-constant Networkscited by 160
  15. 2020Randomized Smoothing of All Shapes and Sizescited by 146
  16. 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
  17. 2019Adversarial Training Can Hurt Generalizationcited by 131
  18. 2020Feature Purification: How Adversarial Training Performs Robust Deep Learningcited by 107
  19. 2019Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagationcited by 87
  20. 2020Learning Long-Term Dependencies in Irregularly-Sampled Time Seriescited by 82
  21. 2018Adversarial examples from computational constraintscited by 79
  22. 2019Adversarially robust transfer learningcited by 66
  23. 2019Instance adaptive adversarial training: Improved accuracy tradeoffs in neural netscited by 55
  24. 2019Risk Averse Robust Adversarial Reinforcement Learningcited by 54
  25. 2019Task-Relevant Adversarial Imitation Learningcited by 45
  26. 2019Are Perceptually-Aligned Gradients a General Property of Robust Classifiers?cited by 42
  27. 2018Adversarial Training for Adverse Conditions: Robust Metric Localisation using Appearance Transfercited by 25
  28. 2020Training Adversarial Agents to Exploit Weaknesses in Deep Control Policiescited by 24
  29. 2020On The Verification of Neural ODEs with Stochastic Guaranteescited by 24
  30. 2019Bridging Adversarial Robustness and Gradient Interpretabilitycited by 22
  31. 2020Adversarially-Trained Deep Nets Transfer Better: Illustration on Image Classificationcited by 18
  32. 2019Adversarial Robustness as a Prior for Learned Representationscited by 17
  33. 2019Interaction-aware Multi-agent Tracking and Probabilistic Behavior Prediction via Adversarial Learningcited by 15
  34. 2020Elaborating on Learned Demonstrations with Temporal Logic Specificationscited by 14
  35. 2020Robust Multiple-Path Orienteering Problem: Securing Against Adversarial Attackscited by 8
  36. 2019Adversarial Feature Training for Generalizable Robotic Visuomotor Controlcited by 7
  37. 2018Safe Motion Planning in Unknown Environments: Optimality Benchmarks and Tractable Policiescited by 1
  38. 2019Active Perception in Adversarial Scenarios using Maximum Entropy Deep Reinforcement Learningcited by 1

Built on by

Cited by year

Citing papers per year: 2021: 9; 2022: 5; 2023: 4; 2024: 1; 2025: 1.

20 papers.

Sort:

By area:

  1. 2021Closed-form Continuous-time Neural Modelscited by 65
  2. 2023Dataset Distillation with Convexified Implicit Gradientscited by 46
  3. 2021Causal Navigation by Continuous-time Neural Networkscited by 23
  4. 2021On-Off Center-Surround Receptive Fields for Accurate and Robust Image Classificationcited by 15
  5. 2022Evolution of Neural Tangent Kernels under Benign and Adversarial Trainingcited by 14
  6. 2021Sparse Flows: Pruning Continuous-depth Modelscited by 13
  7. 2023A Human-Centered Safe Robot Reinforcement Learning Framework with Interactive Behaviorscited by 13
  8. 2021GoTube: Scalable Stochastic Verification of Continuous-Depth Modelscited by 12
  9. 2021Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot Systemscited by 12
  10. 2021BarrierNet: A Safety-Guaranteed Layer for Neural Networkscited by 11
  11. 2022Revisiting the Adversarial Robustness-Accuracy Tradeoff in Robot Learningcited by 6
  12. 2024Adversarial Training: A Surveycited by 6
8 more
  1. 2022Are All Vision Models Created Equal? A Study of the Open-Loop to Closed-Loop Causality Gapcited by 5
  2. 2021Adversarially Regularized Policy Learning Guided by Trajectory Optimizationcited by 4
  3. 2021Interactive Analysis of CNN Robustnesscited by 4
  4. 2022Quantization-aware Interval Bound Propagation for Training Certifiably Robust Quantized Neural Networkscited by 3
  5. 2023Machine Learning Meets Advanced Robotic Manipulationcited by 2
  6. 2022Interpreting Neural Policies with Disentangled Tree Representationscited by 1
  7. 2025Post-Convergence Sim-to-Real Policy Transfer: A Principled Alternative to Cherry-Pickingcited by 1
  8. 2023Robustness Analysis of Continuous-Depth Models with Lagrangian Techniquescited by 0