mlsanityData to 7 Oct 2026

Targeted Adversarial Training for Natural Language Understanding

arXiv 2104.05847 ↗First posted 12 Apr 2021cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

We present a simple yet effective Targeted Adversarial Training (TAT) algorithm to improve adversarial training for natural language understanding. The key idea is to introspect current mistakes and prioritize adversarial training steps to where the model errs the most. Experiments show that TAT can significantly improve accuracy over standard adversarial training on GLUE and attain new state-of-the-art zero-shot results on XNLI. Our code will be released at: https://github.com/namisan/mt-dnn.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 29 of its 46 references that are in the map. First posted on arXiv 12 Apr 2021.

Builds on

29 of 46 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  9. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  10. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  11. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  12. 2018Neural Network Acceptability Judgmentscited by 952
17 more
  1. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  2. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  3. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  4. 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
  5. 2019Adversarial Training for Free!cited by 638
  6. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  7. 2019Lookahead Optimizer: k steps forward, 1 step backcited by 333
  8. 2019SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimizationcited by 253
  9. 2020InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Trainingcited by 237
  10. 2020Better Fine-Tuning by Reducing Representational Collapsecited by 156
  11. 2018Lessons from Natural Language Inference in the Clinical Domaincited by 149
  12. 2020Adversarial Training for Large Neural Language Modelscited by 109
  13. 2019Robust Neural Machine Translation with Doubly Adversarial Inputscited by 105
  14. 2019Visualizing and Understanding the Effectiveness of BERTcited by 67
  15. 2020The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understandingcited by 28
  16. 2020Posterior Differential Regularization with f-divergence for Improving Model Robustnesscited by 20
  17. 2020Adversarial Training for Commonsense Inferencecited by 13

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 2.

5 papers.

  1. 2022A Survey of Adversarial Defences and Robustness in NLPcited by 22
  2. 2021Unsupervised Natural Language Inference Using PHL Triplet Generationcited by 6
  3. 2021Sharpness-Aware Minimization with Dynamic Reweightingcited by 6
  4. 2022Domain Confused Contrastive Learning for Unsupervised Domain Adaptationcited by 4
  5. 2021How Does Adversarial Fine-Tuning Benefit BERT?cited by 3