# New Intent Discovery with Pre-training and Contrastive Learning

arXiv: https://arxiv.org/abs/2205.12914 · HTML: https://mlsanity.com/paper/2205.12914/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 57 references that are in the map.
- First posted on arXiv 25 May 2022.

## Builds on (most cited first, 30 of 40)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [SCAN: Learning to Classify Images without Labels](https://mlsanity.com/paper/2005.12320/) (2020)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://mlsanity.com/paper/2011.01403/) (2020)
- [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://mlsanity.com/paper/2006.03659/) (2020)
- [Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering](https://mlsanity.com/paper/1610.04794/) (2016)
- [ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer](https://mlsanity.com/paper/2105.11741/) (2021)
- [Learning to Discover Novel Visual Categories via Deep Transfer Clustering](https://mlsanity.com/paper/1908.09884/) (2019)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [Learning to cluster in order to transfer across domains and tasks](https://mlsanity.com/paper/1711.10125/) (2017)
- [Self-Guided Contrastive Learning for BERT Sentence Representations](https://mlsanity.com/paper/2106.07345/) (2021)
- [Multi-class Classification without Multi-class Labels](https://mlsanity.com/paper/1901.00544/) (2019)
- [ConveRT: Efficient and Accurate Conversational Representations from Transformers](https://mlsanity.com/paper/1911.03688/) (2019)
- [Supporting Clustering with Contrastive Learning](https://mlsanity.com/paper/2103.12953/) (2021)
- [DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue](https://mlsanity.com/paper/2009.13570/) (2020)
- [Discovering New Intents with Deep Aligned Clustering](https://mlsanity.com/paper/2012.08987/) (2020)

## Built on by (most cited first, 28 of 28)

- [ClusterLLM: Large Language Models as a Guide for Text Clustering](https://mlsanity.com/paper/2305.14871/) (2023)
- [Generalized Category Discovery with Decoupled Prototypical Network](https://mlsanity.com/paper/2211.15115/) (2022)
- [MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversations](https://mlsanity.com/paper/2403.10943/) (2024)
- [IDAS: Intent Discovery with Abstractive Summarization](https://mlsanity.com/paper/2305.19783/) (2023)
- [Cultural Learning-Based Culture Adaptation of Language Models](https://mlsanity.com/paper/2504.02953/) (2025)
- [Watch the Neighbors: A Unified K-Nearest Neighbor Contrastive Learning Framework for OOD Intent Discovery](https://mlsanity.com/paper/2210.08909/) (2022)
- [A Clustering Framework for Unsupervised and Semi-supervised New Intent Discovery](https://mlsanity.com/paper/2304.07699/) (2023)
- [Context-Aware Clustering using Large Language Models](https://mlsanity.com/paper/2405.00988/) (2024)
- [IntentGPT: Few-shot Intent Discovery with Large Language Models](https://mlsanity.com/paper/2411.10670/) (2024)
- [A Diffusion Weighted Graph Framework for New Intent Discovery](https://mlsanity.com/paper/2310.15836/) (2023)
- [GLEAN: Active Generalized Category Discovery with Diverse LLM Feedback](https://mlsanity.com/paper/2502.18414/) (2025)
- [Decoupling Pseudo Label Disambiguation and Representation Learning for Generalized Intent Discovery](https://mlsanity.com/paper/2305.17699/) (2023)
- [Revealing User Familiarity Bias in Task-Oriented Dialogue via Interactive Evaluation](https://mlsanity.com/paper/2305.13857/) (2023)
- [DNA: Denoised Neighborhood Aggregation for Fine-grained Category Discovery](https://mlsanity.com/paper/2310.10151/) (2023)
- [RoNID: New Intent Discovery with Generated-Reliable Labels and Cluster-friendly Representations](https://mlsanity.com/paper/2404.08977/) (2024)
- [Improving Language Models with Intentional Analysis](https://mlsanity.com/paper/2502.04689/) (2025)
- [Analysis of Utterance Embeddings and Clustering Methods Related to Intent Induction for Task-Oriented Dialogue](https://mlsanity.com/paper/2212.02021/) (2022)
- [Multi-Stage Coarse-to-Fine Contrastive Learning for Conversation Intent Induction](https://mlsanity.com/paper/2303.05034/) (2023)
- [Prompt-enhanced Network for Hateful Meme Classification](https://mlsanity.com/paper/2411.07527/) (2024)
- [Unleashing the Potential of Model Bias for Generalized Category Discovery](https://mlsanity.com/paper/2412.12501/) (2024)
- [REVISION:Reflective Intent Mining and Online Reasoning Auxiliary for E-commerce Visual Search System Optimization](https://mlsanity.com/paper/2510.22739/) (2025)
- [Discovering New Intents Using Latent Variables](https://mlsanity.com/paper/2210.11804/) (2022)
- [Multi-View Zero-Shot Open Intent Induction from Dialogues: Multi Domain Batch and Proxy Gradient Transfer](https://mlsanity.com/paper/2303.13099/) (2023)
- [One Stone, Four Birds: A Comprehensive Solution for QA System Using Supervised Contrastive Learning](https://mlsanity.com/paper/2407.09011/) (2024)
- [Pseudo-Label Enhanced Prototypical Contrastive Learning for Uniformed Intent Discovery](https://mlsanity.com/paper/2410.20219/) (2024)
- [Generalized Category Discovery in Event-Centric Contexts: Latent Pattern Mining with LLMs](https://mlsanity.com/paper/2505.23304/) (2025)
- [Evaluating LLMs Without Oracle Feedback: Agentic Annotation Evaluation Through Unsupervised Consistency Signals](https://mlsanity.com/paper/2509.08809/) (2025)
- [Proactive Dialogue Model with Intent Prediction](https://mlsanity.com/paper/2604.27379/) (2026)

## Abstract (arXiv)

New intent discovery aims to uncover novel intent categories from user utterances to expand the set of supported intent classes. It is a critical task for the development and service expansion of a practical dialogue system. Despite its importance, this problem remains under-explored in the literature. Existing approaches typically rely on a large amount of labeled utterances and employ pseudo-labeling methods for representation learning and clustering, which are label-intensive, inefficient, and inaccurate. In this paper, we provide new solutions to two important research questions for new intent discovery: (1) how to learn semantic utterance representations and (2) how to better cluster utterances. Particularly, we first propose a multi-task pre-training strategy to leverage rich unlabeled data along with external labeled data for representation learning. Then, we design a new contrastive loss to exploit self-supervisory signals in unlabeled data for clustering. Extensive experiments on three intent recognition benchmarks demonstrate the high effectiveness of our proposed method, which outperforms state-of-the-art methods by a large margin in both unsupervised and semi-supervised scenarios. The source code will be available at https://github.com/zhang-yu-wei/MTP-CLNN.