mlsanityData to 14 Oct 2026

Removing Undesirable Feature Contributions Using Out-of-Distribution Data

arXiv 2101.06639 ↗First posted 17 Jan 2021cs.LG cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

Several data augmentation methods deploy unlabeled-in-distribution (UID) data to bridge the gap between the training and inference of neural networks. However, these methods have clear limitations in terms of availability of UID data and dependence of algorithms on pseudo-labels. Herein, we propose a data augmentation method to improve generalization in both adversarial and standard learning by using out-of-distribution (OOD) data that are devoid of the abovementioned issues. We show how to improve generalization theoretically using OOD data in each learning scenario and complement our theoretical analysis with experiments on CIFAR-10, CIFAR-100, and a subset of ImageNet. The results indicate that undesirable features are shared even among image data that seem to have little correlation from a human point of view. We also present the advantages of the proposed method through comparison with other data augmentation methods, which can be used in the absence of UID data. Furthermore, we demonstrate that the proposed method can further improve the existing state-of-the-art adversarial training.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 38 of its 54 references that are in the map. First posted on arXiv 17 Jan 2021.

Builds on

38 of 54 references.

References · 38

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  4. 2013Intriguing properties of neural networkscited by 5,155
  5. 2016Wide Residual Networkscited by 3,070
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  7. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  8. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  9. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  10. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  11. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  12. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
26 more
  1. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  2. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  3. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  4. 2018Deep Anomaly Detection with Outlier Exposurecited by 942
  5. 2018Robustness May Be at Odds with Accuracycited by 880
  6. 2016Universal adversarial perturbationscited by 784
  7. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  8. 2017VisDA: The Visual Domain Adaptation Challengecited by 435
  9. 2019Unlabeled Data Improves Adversarial Robustnesscited by 416
  10. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  11. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  12. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 334
  13. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  14. 2018Adversarially Robust Generalization Requires More Datacited by 318
  15. 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
  16. 2018Adversarial Logit Pairingcited by 224
  17. 2019Learning De-biased Representations with Biased Representationscited by 199
  18. 2019Are Labels Required for Improving Adversarial Robustness?cited by 163
  19. 2017Generative Adversarial Perturbationscited by 153
  20. 2019Cross-Domain Transferability of Adversarial Perturbationscited by 92
  21. 2017Good Semi-supervised Learning that Requires a Bad GANcited by 86
  22. 2020Adversarial Vertex Mixup: Toward Better Adversarially Robust Generalizationcited by 73
  23. 2020Adversarial Robustness on In- and Out-Distribution Improves Explainabilitycited by 73
  24. 2019Robustness to Adversarial Perturbations in Learning from Incomplete Datacited by 49
  25. 2019What it Thinks is Important is Important: Robustness Transfers through Input Gradientscited by 34
  26. 2020Hold me tight! Influence of discriminative features on deep network boundariescited by 31

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 7; 2023: 7; 2024: 3; 2025: 2.

23 papers.

Sort:

By area:

  1. 2022Dataset Condensation with Contrastive Signalscited by 105
  2. 2021Open-set Label Noise Can Improve Robustness Against Inherent Label Noisecited by 59
  3. 2022Weakly Supervised Semantic Segmentation using Out-of-Distribution Datacited by 58
  4. 2023OpenMix: Exploring Outlier Samples for Misclassification Detectioncited by 38
  5. 2022Open-Sampling: Exploring Out-of-Distribution data for Re-balancing Long-tailed datasetscited by 25
  6. 2022Meta-Query-Net: Resolving Purity-Informativeness Dilemma in Open-set Active Learningcited by 25
  7. 2022Igeood: An Information Geometry Approach to Out-of-Distribution Detectioncited by 20
  8. 2023On the Effectiveness of Out-of-Distribution Data in Self-Supervised Long-Tail Learningcited by 20
  9. 2021Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Featurescited by 14
  10. 2023Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detectioncited by 14
  11. 2024DAFA: Distance-Aware Fair Adversarial Trainingcited by 11
  12. 2024Understanding and Mitigating Miscalibration in Prompt Tuning for Vision-Language Modelscited by 9
11 more
  1. 2021Domain Invariant Adversarial Learningcited by 7
  2. 2023Splitting the Difference on Adversarial Trainingcited by 5
  3. 2021Can Targeted Adversarial Examples Transfer When the Source and Target Models Have No Label Space Overlap?cited by 3
  4. 2023Out-Of-Domain Unlabeled Data Improves Generalizationcited by 3
  5. 2023A Functional Data Perspective and Baseline On Multi-Layer Out-of-Distribution Detectioncited by 2
  6. 2023Adaptive Shortcut Debiasing for Online Continual Learningcited by 2
  7. 2022On the Robustness and Anomaly Detection of Sparse Neural Networkscited by 1
  8. 2022Inducing Data Amplification Using Auxiliary Datasets in Adversarial Trainingcited by 1
  9. 2024Prioritizing Informative Features and Examples for Deep Learning from Noisy Datacited by 0
  10. 2025Unsupervised Contrastive Learning Using Out-Of-Distribution Data for Long-Tailed Datasetcited by 0
  11. 2025Out-of-distribution data supervision towards biomedical semantic segmentationcited by 0