mlsanityData to 14 Oct 2026

Impact of Adversarial Training on Robustness and Generalizability of Language Models

arXiv 2211.05523 ↗First posted 10 Nov 2022cs.CL cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Adversarial training is widely acknowledged as the most effective defense against adversarial attacks. However, it is also well established that achieving both robustness and generalization in adversarially trained models involves a trade-off. The goal of this work is to provide an in depth comparison of different approaches for adversarial training in language models. Specifically, we study the effect of pre-training data augmentation as well as training time input perturbations vs. embedding space perturbations on the robustness and generalization of transformer-based language models. Our findings suggest that better robustness can be achieved by pre-training data augmentation or by training with input space perturbation. However, training with embedding space perturbation significantly improves generalization. A linguistic correlation analysis of neurons of the learned models reveals that the improved generalization is due to 'more specialized' neurons. To the best of our knowledge, this is the first work to carry out a deep qualitative analysis of different methods of generating adversarial examples in adversarial training of language models.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 45 of its 56 references that are in the map. First posted on arXiv 10 Nov 2022.

Builds on

45 of 56 references.

References · 45

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  8. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  9. 2013Intriguing properties of neural networkscited by 5,155
  10. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  11. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  12. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
33 more
  1. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  2. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 920
  3. 2018Robustness May Be at Odds with Accuracycited by 880
  4. 2022A Generalist Agentcited by 808
  5. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
  6. 2015The Limitations of Deep Learning in Adversarial Settingscited by 790
  7. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
  8. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
  9. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 491
  10. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 457
  11. 2018Generating Natural Language Adversarial Examplescited by 424
  12. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 383
  13. 2020Large-Scale Adversarial Training for Vision-and-Language Representation Learningcited by 327
  14. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  15. 2018Certified Defenses against Adversarial Examplescited by 306
  16. 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 172
  17. 2019Certified Robustness to Adversarial Word Substitutionscited by 168
  18. 2017What do Neural Machine Translation Models Learn about Morphology?cited by 166
  19. 2016Counter-fitting Word Vectors to Linguistic Constraintscited by 140
  20. 2018What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Modelscited by 124
  21. 2020Adversarial Training for Large Neural Language Modelscited by 111
  22. 2020Analyzing Individual Neurons in Pre-trained Language Modelscited by 97
  23. 2020SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustnesscited by 96
  24. 2019Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagationcited by 88
  25. 2021Neuron-level Interpretation of Deep NLP Models: A Surveycited by 81
  26. 2021Towards Improving Adversarial Training of NLP Modelscited by 79
  27. 2020Robustness Verification for Transformerscited by 76
  28. 2020SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutionscited by 73
  29. 2021Certified Robustness to Text Adversarial Attacks by Randomized [MASK]cited by 57
  30. 2021Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitutioncited by 56
  31. 2022Discovering Latent Concepts Learned in BERTcited by 56
  32. 2021Improved Text Classification via Contrastive Adversarial Trainingcited by 38
  33. 2021Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack Frameworkcited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 2; 2025: 4; 2026: 1.

9 papers.

  1. 2023Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibilitycited by 30
  2. 2025Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metricscited by 12
  3. 2022A3T: Accuracy Aware Adversarial Trainingcited by 4
  4. 2024Finding a Needle in the Adversarial Haystack: A Targeted Paraphrasing Approach For Uncovering Edge Cases with Minimal Distribution Distortioncited by 3
  5. 2024Explaining the role of Intrinsic Dimensionality in Adversarial Trainingcited by 2
  6. 2025Detecting and Pruning Prominent but Detrimental Neurons in Large Language Modelscited by 2
  7. 2025Adversarial Training Improves Generalization Under Distribution Shifts in Bird Sound Classificationcited by 0
  8. 2025A Robust and Explainable Transformer-Based Framework for Phishing Email Detectioncited by 0
  9. 2026Private Vertical Federated Inference for Time-Seriescited by 0