# Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning

arXiv: https://arxiv.org/abs/2105.14289 · HTML: https://mlsanity.com/paper/2105.14289/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 34 references that are in the map.
- First posted on arXiv 29 May 2021.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Likelihood Ratios for Out-of-Distribution Detection](https://mlsanity.com/paper/1906.02845/) (2019)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Additive Margin Softmax for Face Verification](https://mlsanity.com/paper/1801.05599/) (2018)
- [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://mlsanity.com/paper/2011.01403/) (2020)
- [DOC: Deep Open Classification of Text Documents](https://mlsanity.com/paper/1709.08716/) (2017)
- [From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots](https://mlsanity.com/paper/1801.01957/) (2018)
- [Deep Unknown Intent Detection with Margin Loss](https://mlsanity.com/paper/1906.00434/) (2019)
- [Out-of-domain Detection for Natural Language Understanding in Dialog Systems](https://mlsanity.com/paper/1909.03862/) (2019)
- [Generating Fluent Adversarial Examples for Natural Languages](https://mlsanity.com/paper/2007.06174/) (2020)
- [Adversarial Diversity and Hard Positive Generation](https://mlsanity.com/paper/1605.01775/) (2016)
- [Joint Learning of Domain Classification and Out-of-Domain Detection with Dynamic Class Weighting for Satisficing False Acceptance Rates](https://mlsanity.com/paper/1807.00072/) (2018)
- [Chat Detection in an Intelligent Assistant: Combining Task-oriented and Non-task-oriented Spoken Dialogue Systems](https://mlsanity.com/paper/1705.00746/) (2017)

## Built on by (most cited first, 30 of 42)

- [POEM: Out-of-Distribution Detection with Posterior Sampling](https://mlsanity.com/paper/2206.13687/) (2022)
- [A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation](https://mlsanity.com/paper/2305.11391/) (2023)
- [Rank-N-Contrast: Learning Continuous Representations for Regression](https://mlsanity.com/paper/2210.01189/) (2022)
- [Contrastive Positive Sample Propagation along the Audio-Visual Event Line](https://mlsanity.com/paper/2211.09980/) (2022)
- [A Survey on Out-of-Distribution Detection in NLP](https://mlsanity.com/paper/2305.03236/) (2023)
- [Is Fine-tuning Needed? Pre-trained Language Models Are Near Perfect for Out-of-Domain Detection](https://mlsanity.com/paper/2305.13282/) (2023)
- [Graph Enhanced Contrastive Learning for Radiology Findings Summarization](https://mlsanity.com/paper/2204.00203/) (2022)
- [Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection](https://mlsanity.com/paper/2402.17256/) (2024)
- [Estimating Soft Labels for Out-of-Domain Intent Detection](https://mlsanity.com/paper/2211.05561/) (2022)
- [Multi-Level Knowledge Distillation for Out-of-Distribution Detection in Text](https://mlsanity.com/paper/2211.11300/) (2022)
- [CLOC: Contrastive Learning for Ordinal Classification with Multi-Margin N-pair Loss](https://mlsanity.com/paper/2504.17813/) (2025)
- [Generalized Intent Discovery: Learning from Open World Dialogue System](https://mlsanity.com/paper/2209.06030/) (2022)
- [LMC: Large Model Collaboration with Cross-assessment for Training-Free Open-Set Object Recognition](https://mlsanity.com/paper/2309.12780/) (2023)
- [Watch the Neighbors: A Unified K-Nearest Neighbor Contrastive Learning Framework for OOD Intent Discovery](https://mlsanity.com/paper/2210.08909/) (2022)
- [Enhancing Out-of-Distribution Detection in Natural Language Understanding via Implicit Layer Ensemble](https://mlsanity.com/paper/2210.11034/) (2022)
- [Distribution Calibration for Out-of-Domain Detection with Bayesian Approximation](https://mlsanity.com/paper/2209.06612/) (2022)
- [Out-of-Domain Intent Detection Considering Multi-Turn Dialogue Contexts](https://mlsanity.com/paper/2305.03237/) (2023)
- [UniNL: Aligning Representation Learning with Scoring Function for OOD Detection via Unified Neighborhood Learning](https://mlsanity.com/paper/2210.10722/) (2022)
- [Decoupling Pseudo Label Disambiguation and Representation Learning for Generalized Intent Discovery](https://mlsanity.com/paper/2305.17699/) (2023)
- [Continual Generalized Intent Discovery: Marching Towards Dynamic and Open-world Intent Recognition](https://mlsanity.com/paper/2310.10184/) (2023)
- [Distinguish Sense from Nonsense: Out-of-Scope Detection for Virtual Assistants](https://mlsanity.com/paper/2301.06544/) (2023)
- [MSAC: Multiple Speech Attribute Control Method for Reliable Speech Emotion Recognition](https://mlsanity.com/paper/2308.04025/) (2023)
- [SELFOOD: Self-Supervised Out-Of-Distribution Detection via Learning to Rank](https://mlsanity.com/paper/2305.14696/) (2023)
- [APP: Adaptive Prototypical Pseudo-Labeling for Few-shot OOD Detection](https://mlsanity.com/paper/2310.13380/) (2023)
- [The Open-World Lottery Ticket Hypothesis for OOD Intent Classification](https://mlsanity.com/paper/2210.07071/) (2022)
- [Disentangling Confidence Score Distribution for Out-of-Domain Intent Detection with Energy-Based Learning](https://mlsanity.com/paper/2210.08830/) (2022)
- [Probing Out-of-Distribution Robustness of Language Models with Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2301.11660/) (2023)
- [Universal Domain Adaptation for Robust Handling of Distributional Shifts in NLP](https://mlsanity.com/paper/2310.14849/) (2023)
- [DualTalker: A Cross-Modal Dual Learning Approach for Speech-Driven 3D Facial Animation](https://mlsanity.com/paper/2311.04766/) (2023)
- [From Zero to Hero: Cold-Start Anomaly Detection](https://mlsanity.com/paper/2405.20341/) (2024)

## Abstract (arXiv)

Detecting Out-of-Domain (OOD) or unknown intents from user queries is essential in a task-oriented dialog system. A key challenge of OOD detection is to learn discriminative semantic features. Traditional cross-entropy loss only focuses on whether a sample is correctly classified, and does not explicitly distinguish the margins between categories. In this paper, we propose a supervised contrastive learning objective to minimize intra-class variance by pulling together in-domain intents belonging to the same class and maximize inter-class variance by pushing apart samples from different classes. Besides, we employ an adversarial augmentation mechanism to obtain pseudo diverse views of a sample in the latent space. Experiments on two public datasets prove the effectiveness of our method capturing discriminative representations for OOD detection.