# LANID: LLM-assisted New Intent Discovery

arXiv: https://arxiv.org/abs/2503.23740 · HTML: https://mlsanity.com/paper/2503.23740/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 60 references that are in the map.
- First posted on arXiv 31 Mar 2025.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [Yara Parser: A Fast and Accurate Dependency Parser](https://mlsanity.com/paper/1503.06733/) (2015)
- [Multi-class Classification without Multi-class Labels](https://mlsanity.com/paper/1901.00544/) (2019)
- [Discovering New Intents with Deep Aligned Clustering](https://mlsanity.com/paper/2012.08987/) (2020)
- [Discovering New Intents via Constrained Deep Adaptive Clustering with Cluster Refinement](https://mlsanity.com/paper/1911.08891/) (2019)
- [Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training](https://mlsanity.com/paper/2106.08616/) (2021)
- [ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?](https://mlsanity.com/paper/2306.01386/) (2023)
- [A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding](https://mlsanity.com/paper/2304.04256/) (2023)
- [Effectiveness of Pre-training for Few-shot Intent Classification](https://mlsanity.com/paper/2109.05782/) (2021)
- [Intent Mining from past conversations for conversational agent](https://mlsanity.com/paper/2005.11014/) (2020)
- [TEXTOIR: An Integrated and Visualized Platform for Text Open Intent Recognition](https://mlsanity.com/paper/2110.15063/) (2021)
- [Dialog Intent Induction with Deep Multi-View Clustering](https://mlsanity.com/paper/1908.11487/) (2019)
- [Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropization](https://mlsanity.com/paper/2205.07208/) (2022)
- [Large Language Models Meet Open-World Intent Discovery and Recognition: An Evaluation of ChatGPT](https://mlsanity.com/paper/2310.10176/) (2023)
- [Estimating Soft Labels for Out-of-Domain Intent Detection](https://mlsanity.com/paper/2211.05561/) (2022)
- [A Clustering Framework for Unsupervised and Semi-supervised New Intent Discovery](https://mlsanity.com/paper/2304.07699/) (2023)
- [Dialog Intent Induction via Density-based Deep Clustering Ensemble](https://mlsanity.com/paper/2201.06731/) (2022)

## Built on by (most cited first, 3 of 3)

- [Toward Super Agent System with Hybrid AI Routers](https://mlsanity.com/paper/2504.10519/) (2025)
- [QUADS: QUAntized Distillation Framework for Efficient Speech Language Understanding](https://mlsanity.com/paper/2505.14723/) (2025)
- [NILC: Discovering New Intents with LLM-assisted Clustering](https://mlsanity.com/paper/2511.05913/) (2025)

## Abstract (arXiv)

Task-oriented Dialogue Systems (TODS) often face the challenge of encountering new intents. New Intent Discovery (NID) is a crucial task that aims to identify these novel intents while maintaining the capability to recognize existing ones. Previous efforts to adapt TODS to new intents have struggled with inadequate semantic representation or have depended on external knowledge, which is often not scalable or flexible. Recently, Large Language Models (LLMs) have demonstrated strong zero-shot capabilities; however, their scale can be impractical for real-world applications that involve extensive queries. To address the limitations of existing NID methods by leveraging LLMs, we propose LANID, a framework that enhances the semantic representation of lightweight NID encoders with the guidance of LLMs. Specifically, LANID employs the K-nearest neighbors and Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithms to sample selective utterance pairs from the training set. It then queries an LLM to ascertain the relationships between these pairs. The data produced from this process is utilized to design a contrastive fine-tuning task, which is then used to train a small encoder with a contrastive triplet loss. Our experimental results demonstrate the efficacy of the proposed method across three distinct NID datasets, surpassing strong baselines in both unsupervised and semi-supervised settings. Our code is available at https://github.com/floatSDSDS/LANID.