# Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustness

arXiv: https://arxiv.org/abs/2203.07653 · HTML: https://mlsanity.com/paper/2203.07653/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 82 references that are in the map.
- First posted on arXiv 15 Mar 2022.

## Builds on (most cited first, 30 of 62)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://mlsanity.com/paper/1902.01007/) (2019)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [From Recognition to Cognition: Visual Commonsense Reasoning](https://mlsanity.com/paper/1811.10830/) (2018)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [Dynabench: Rethinking Benchmarking in NLP](https://mlsanity.com/paper/2104.14337/) (2021)
- [Measuring Robustness to Natural Distribution Shifts in Image Classification](https://mlsanity.com/paper/2007.00644/) (2020)
- [Generalizing to Unseen Domains via Adversarial Data Augmentation](https://mlsanity.com/paper/1805.12018/) (2018)
- [Generating Natural Language Adversarial Examples](https://mlsanity.com/paper/1804.07998/) (2018)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)

## Built on by (most cited first, 23 of 23)

- [ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasets](https://mlsanity.com/paper/2209.00613/) (2022)
- [Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models](https://mlsanity.com/paper/2310.00836/) (2023)
- [Evaluating Human-AI Collaboration: A Review and Methodological Framework](https://mlsanity.com/paper/2407.19098/) (2024)
- [Adversarial Bayesian Augmentation for Single-Source Domain Generalization](https://mlsanity.com/paper/2307.09520/) (2023)
- [A Survey on Out-of-Distribution Evaluation of Neural NLP Models](https://mlsanity.com/paper/2306.15261/) (2023)
- [Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance](https://mlsanity.com/paper/2508.19764/) (2025)
- [BioTABQA: Instruction Learning for Biomedical Table Question Answering](https://mlsanity.com/paper/2207.02419/) (2022)
- [A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities](https://mlsanity.com/paper/2210.08906/) (2022)
- [A Study on the Efficiency and Generalization of Light Hybrid Retrievers](https://mlsanity.com/paper/2210.01371/) (2022)
- [Data Optimization in Deep Learning: A Survey](https://mlsanity.com/paper/2310.16499/) (2023)
- [Out-of-Distribution Data: An Acquaintance of Adversarial Examples -- A Survey](https://mlsanity.com/paper/2404.05219/) (2024)
- [Improving Shift Invariance in Convolutional Neural Networks with Translation Invariant Polyphase Sampling](https://mlsanity.com/paper/2404.07410/) (2024)
- [Domain Generalization via Causal Adjustment for Cross-Domain Sentiment Analysis](https://mlsanity.com/paper/2402.14536/) (2024)
- [Segmentation Consistency Training: Out-of-Distribution Generalization for Medical Image Segmentation](https://mlsanity.com/paper/2205.15428/) (2022)
- [Pretrained Transformers Do not Always Improve Robustness](https://mlsanity.com/paper/2210.07663/) (2022)
- [Neural Retriever and Go Beyond: A Thesis Proposal](https://mlsanity.com/paper/2205.16005/) (2022)
- [A Survey of Parameters Associated with the Quality of Benchmarks in NLP](https://mlsanity.com/paper/2210.07566/) (2022)
- [Out-of-Distribution Generalization in Text Classification: Past, Present, and Future](https://mlsanity.com/paper/2305.14104/) (2023)
- [Estimating Semantic Similarity between In-Domain and Out-of-Domain Samples](https://mlsanity.com/paper/2306.01206/) (2023)
- [Stability Evaluation via Distributional Perturbation Analysis](https://mlsanity.com/paper/2405.03198/) (2024)
- [Robust Source-Free Domain Adaptation for Fundus Image Segmentation](https://mlsanity.com/paper/2310.16665/) (2023)
- [IMO: Greedy Layer-Wise Sparse Representation Learning for Out-of-Distribution Text Classification with Pre-trained Models](https://mlsanity.com/paper/2404.13504/) (2024)
- [Probabilistic Runtime Verification, Evaluation and Risk Assessment of Visual Deep Learning Systems](https://mlsanity.com/paper/2509.19419/) (2025)

## Abstract (arXiv)

Data modification, either via additional training datasets, data augmentation, debiasing, and dataset filtering, has been proposed as an effective solution for generalizing to out-of-domain (OOD) inputs, in both natural language processing and computer vision literature. However, the effect of data modification on adversarial robustness remains unclear. In this work, we conduct a comprehensive study of common data modification strategies and evaluate not only their in-domain and OOD performance, but also their adversarial robustness (AR). We also present results on a two-dimensional synthetic dataset to visualize the effect of each method on the training distribution. This work serves as an empirical study towards understanding the relationship between generalizing to unseen domains and defending against adversarial perturbations. Our findings suggest that more data (either via additional datasets or data augmentation) benefits both OOD accuracy and AR. However, data filtering (previously shown to improve OOD accuracy on natural language inference) hurts OOD accuracy on other tasks such as question answering and image classification. We provide insights from our experiments to inform future work in this direction.