mlsanityData to 7 Oct 2026

Fine-Tuning Deteriorates General Textual Out-of-Distribution Detection by Distorting Task-Agnostic Features

arXiv 2301.12715 ↗First posted 30 Jan 2023cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Detecting out-of-distribution (OOD) inputs is crucial for the safe deployment of natural language processing (NLP) models. Though existing methods, especially those based on the statistics in the feature space of fine-tuned pre-trained language models (PLMs), are claimed to be effective, their effectiveness on different types of distribution shifts remains underexplored. In this work, we take the first step to comprehensively evaluate the mainstream textual OOD detection methods for detecting semantic and non-semantic shifts. We find that: (1) no existing method behaves well in both settings; (2) fine-tuning PLMs on in-distribution data benefits detecting semantic shifts but severely deteriorates detecting non-semantic shifts, which can be attributed to the distortion of task-agnostic features. To alleviate the issue, we present a simple yet effective general OOD score named GNOME that integrates the confidence scores derived from the task-agnostic and task-specific representations. Experiments show that GNOME works well in both semantic and non-semantic shift scenarios, and further brings significant improvement on two cross-task benchmarks where both kinds of shifts simultaneously take place. Our code is available at https://github.com/lancopku/GNOME.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 40 of its 72 references that are in the map. First posted on arXiv 30 Jan 2023.

Builds on

40 of 72 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  7. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  8. 2020Supervised Contrastive Learningcited by 2,803
  9. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  10. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  11. 2016Concrete Problems in AI Safetycited by 1,582
  12. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
28 more
  1. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  2. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  3. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  4. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  5. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  6. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  7. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  8. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  9. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 378
  10. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  11. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 293
  12. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  13. 2018Large Scale Crowdsourcing and Characterization of Twitter Abusive Behaviorcited by 265
  14. 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 198
  15. 2020Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgettingcited by 169
  16. 2020Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encodercited by 145
  17. 2021Types of Out-of-Distribution Texts and How to Detect Themcited by 80
  18. 2020SENTRY: Selective Entropy Optimization via Committee Consistency for Unsupervised Domain Adaptationcited by 78
  19. 2021Contrastive Out-of-Distribution Detection for Pretrained Transformerscited by 70
  20. 2021Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detectioncited by 67
  21. 2019Deep Unknown Intent Detection with Margin Losscited by 66
  22. 2019Detecting semantic anomaliescited by 50
  23. 2020Feature Space Singularity for Out-of-Distribution Detectioncited by 49
  24. 2019Likelihood Ratios and Generative Classifiers for Unsupervised Out-of-Domain Detection In Task Oriented Dialogcited by 33
  25. 2021Unsupervised Out-of-Domain Detection via Pre-trained Transformerscited by 31
  26. 2021Enhancing the Generalization for Intent Classification and Out-of-Domain Detection in SLUcited by 22
  27. 2021kFolden: k-Fold Ensemble for Out-Of-Distribution Detectioncited by 10
  28. 2022Holistic Sentence Embeddings for Better Out-of-Distribution Detectioncited by 6

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 6; 2025: 2.

12 papers.

  1. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  2. 2024Characterizing LLM Abstention Behavior in Science QA with Context Perturbationscited by 26
  3. 2024BadActs: A Universal Backdoor Defense in the Activation Spacecited by 19
  4. 2025Training-Free Policy Violation Detection via Activation-Space Whitening in LLMscited by 6
  5. 2023Classical Out-of-Distribution Detection Methods Benchmark in Text Classification Taskscited by 5
  6. 2023FLatS: Principled Out-of-Distribution Detection with Feature-Based Likelihood Ratio Scorecited by 5
  7. 2024A new approach for fine-tuning sentence transformers for intent classification and out-of-scope detection taskscited by 4
  8. 2024Out-of-Distribution Detection through Soft Clustering with Non-Negative Kernel Regressioncited by 3
  9. 2024Counterfactuals As a Means for Evaluating Faithfulness of Attribution Methods in Autoregressive Language Modelscited by 3
  10. 2025Merging Smarter, Generalizing Better: Enhancing Model Merging on OOD Datacited by 3
  11. 2024Out-of-Distribution Detection with Attention Head Masking for Multimodal Document Classificationcited by 2
  12. 2023Estimating Semantic Similarity between In-Domain and Out-of-Domain Samplescited by 1