mlsanityData to 14 Oct 2026

Robust Machine Comprehension Models via Adversarial Training

arXiv 1804.06473 ↗First posted 17 Apr 2018cs.CLcited by 44

Not on the map. Why?

Abstract (arXiv)

It is shown that many published models for the Stanford Question Answering Dataset (Rajpurkar et al., 2016) lack robustness, suffering an over 50% decrease in F1 score during adversarial evaluation based on the AddSent (Jia and Liang, 2017) algorithm. It has also been shown that retraining models on data generated by AddSent has limited effect on their robustness. We propose a novel alternative adversary-generation algorithm, AddSentDiverse, that significantly increases the variance within the adversarial training data by providing effective examples that punish the model for making certain superficial assumptions. Further, in order to improve robustness to AddSent's semantic perturbations (e.g., antonyms), we jointly improve the model's semantic-relationship learning capabilities in addition to our AddSentDiverse-based adversarial training data augmentation. With these additions, we show that we can make a state-of-the-art model significantly more robust, achieving a 36.5% increase in F1 score under many different types of adversarial evaluation while maintaining performance on the regular SQuAD task.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 16; 2022: 8; 2023: 13; 2024: 6; 2025: 0; 2026: 1.

44 papers.

Sort:

By area:

  1. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 194
  2. 2021An Empirical Survey of Data Augmentation for Limited Data Learning in NLPcited by 101
  3. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 93
  4. 2021CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understandingcited by 50
  5. 2021Can NLI Models Verify QA Systems' Predictions?cited by 46
  6. 2021HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalizationcited by 32
  7. 2022A Survey of Adversarial Defences and Robustness in NLPcited by 23
  8. 2021Certified Robustness to Programmable Transformations in LSTMscited by 18
  9. 2022Debiasing Masks: A New Framework for Shortcut Mitigation in NLUcited by 16
  10. 2022Recent Advances in Neural Text Generation: A Task-Agnostic Surveycited by 14
  11. 2021Contrastive Fine-tuning Improves Robustness for Neural Rankerscited by 12
  12. 2021Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extractioncited by 11
32 more
  1. 2024PIDformer: Transformer Meets Control Theorycited by 11
  2. 2023Improving Conversational Recommendation Systems via Counterfactual Data Simulationcited by 10
  3. 2023Model-tuning Via Prompts Makes NLP Models Adversarially Robustcited by 9
  4. 2022Residue-Based Natural Language Adversarial Attack Detectioncited by 8
  5. 2021Improving the robustness and accuracy of biomedical language models through adversarial trainingcited by 7
  6. 2024Desiderata for the Context Use of Question Answering Systemscited by 7
  7. 2021Are Multilingual BERT models robust? A Case Study on Adversarial Attacks for Multilingual Question Answeringcited by 6
  8. 2023LEAP: Efficient and Automated Test Method for NLP Softwarecited by 6
  9. 2021Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Modelscited by 5
  10. 2021Enhancing Model Robustness By Incorporating Adversarial Knowledge Into Semantic Representationcited by 4
  11. 2021Reasoning Chain Based Adversarial Attack for Multi-hop Question Answeringcited by 4
  12. 2022A Survey on Measuring and Mitigating Reasoning Shortcuts in Machine Reading Comprehensioncited by 4
  13. 2023The Impacts of Unanswerable Questions on the Robustness of Machine Reading Comprehension Modelscited by 4
  14. 2021Towards Confident Machine Reading Comprehensioncited by 3
  15. 2023Adversary for Social Good: Leveraging Adversarial Attacks to Protect Personal Attribute Privacycited by 3
  16. 2021Model Agnostic Answer Reranking System for Adversarial Question Answeringcited by 2
  17. 2021Improving Robustness of Malware Classifiers using Adversarial Strings Generated from Perturbed Latent Representationscited by 2
  18. 2023Data Augmentation for Neural NLPcited by 2
  19. 2023Mitigating Shortcuts in Language Models with Soft Label Encodingcited by 2
  20. 2024The Impact of Quantization on the Robustness of Transformer-based Text Classifierscited by 2
  21. 2024Specification Overfitting in Artificial Intelligencecited by 2
  22. 2021CLLD: Contrastive Learning with Label Distance for Text Classificationcited by 1
  23. 2022Using calibrator to improve robustness in Machine Reading Comprehensioncited by 0
  24. 2022Improving Lexical Embeddings for Robust Question Answeringcited by 0
  25. 2022An Understanding-Oriented Robust Machine Reading Comprehension Modelcited by 0
  26. 2023Evaluating the Robustness of Machine Reading Comprehension Models to Low Resource Entity Renamingcited by 0
  27. 2023SCAT: Robust Self-supervised Contrastive Learning via Adversarial Training for Text Classificationcited by 0
  28. 2023No offence, Bert -- I insult only humans! Multiple addressees sentence-level attack on toxicity detection neural networkcited by 0
  29. 2023A Lightweight Method to Generate Unanswerable Questions in Englishcited by 0
  30. 2024Contextual Breach: Assessing the Robustness of Transformer-based QA Modelscited by 0
  31. 2024Towards Robust Extractive Question Answering Models: Rethinking the Training Methodologycited by 0
  32. 2026Operational Robustness of LLMs on Code Generationcited by 0