Understanding, Detecting, and Separating Out-of-Distribution Samples and Adversarial Samples in Text Classification
Not on the map. Why?
Abstract (arXiv)
In this paper, we study the differences and commonalities between statistically out-of-distribution (OOD) samples and adversarial (Adv) samples, both of which hurting a text classification model's performance. We conduct analyses to compare the two types of anomalies (OOD and Adv samples) with the in-distribution (ID) ones from three aspects: the input features, the hidden representations in each layer of the model, and the output probability distributions of the classifier. We find that OOD samples expose their aberration starting from the first layer, while the abnormalities of Adv samples do not emerge until the deeper layers of the model. We also illustrate that the models' output probabilities for Adv samples tend to be more unconfident. Based on our observations, we propose a simple method to separate ID, OOD, and Adv samples using the hidden representations and output probabilities of the model. On multiple combinations of ID, OOD datasets, and Adv attacks, our proposed method shows exceptional results on distinguishing ID, OOD, and Adv samples.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 18 of its 37 references that are in the map. First posted on arXiv 9 Apr 2022.
Builds on
18 of 37 references.
References · 18
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
- 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
- 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
- 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
- 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
- 2021Datasets: A Community Library for Natural Language Processingcited by 526
- 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
- 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
- 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 293
- 2019Combating Adversarial Misspellings with Robust Word Recognitioncited by 162
6 more
- 2020Reevaluating Adversarial Examples in Natural Languagecited by 83
- 2019Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classificationcited by 55
- 2020Generating Natural Language Attacks in a Hard Label Black Box Settingcited by 50
- 2022Detection of Word Adversarial Examples in Text Classification: Benchmark and Baseline via Robust Density Estimationcited by 32
- 2021Unsupervised Out-of-Domain Detection via Pre-trained Transformerscited by 31
- 2020Defense of Word-level Adversarial Attacks via Random Substitution Encodingcited by 9
Built on by
Citing papers per year: 2023: 1.
1 paper.