# Large Language Models Meet Open-World Intent Discovery and Recognition: An Evaluation of ChatGPT

arXiv: https://arxiv.org/abs/2310.10176 · HTML: https://mlsanity.com/paper/2310.10176/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 35 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Mathematical Capabilities of ChatGPT](https://mlsanity.com/paper/2301.13867/) (2023)
- [On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2302.12095/) (2023)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine](https://mlsanity.com/paper/2301.08745/) (2023)
- [Unlimiformer: Long-Range Transformers with Unlimited Length Input](https://mlsanity.com/paper/2305.01625/) (2023)
- [Discovering New Intents with Deep Aligned Clustering](https://mlsanity.com/paper/2012.08987/) (2020)
- [Discovering New Intents via Constrained Deep Adaptive Clustering with Cluster Refinement](https://mlsanity.com/paper/1911.08891/) (2019)
- [On the Educational Impact of ChatGPT: Is Artificial Intelligence Ready to Obtain a University Degree?](https://mlsanity.com/paper/2303.11146/) (2023)
- [Revisit Input Perturbation Problems for LLMs: A Unified Robustness Evaluation Framework for Noisy Slot Filling Task](https://mlsanity.com/paper/2310.06504/) (2023)
- [Semi-Supervised Knowledge-Grounded Pre-training for Task-Oriented Dialog Systems](https://mlsanity.com/paper/2210.08873/) (2022)
- [Generalized Intent Discovery: Learning from Open World Dialogue System](https://mlsanity.com/paper/2209.06030/) (2022)
- [PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling](https://mlsanity.com/paper/2208.11508/) (2022)
- [Watch the Neighbors: A Unified K-Nearest Neighbor Contrastive Learning Framework for OOD Intent Discovery](https://mlsanity.com/paper/2210.08909/) (2022)
- [Revisit Out-Of-Vocabulary Problem for Slot Filling: A Unified Contrastive Frameword with Multi-level Data Augmentations](https://mlsanity.com/paper/2302.13584/) (2023)
- [Towards Robust and Generalizable Training: An Empirical Study of Noisy Slot Filling for Input Perturbations](https://mlsanity.com/paper/2310.03518/) (2023)

## Built on by (most cited first, 16 of 16)

- [Thought-Augmented Planning for LLM-Powered Interactive Recommender Agent](https://mlsanity.com/paper/2506.23485/) (2025)
- [Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection](https://mlsanity.com/paper/2402.17256/) (2024)
- [DolphCoder: Echo-Locating Code Large Language Models with Diverse and Multi-Objective Instruction Tuning](https://mlsanity.com/paper/2402.09136/) (2024)
- [Dial-In LLM: Human-Aligned LLM-in-the-loop Intent Clustering for Customer Service Dialogues](https://mlsanity.com/paper/2412.09049/) (2024)
- [Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Models](https://mlsanity.com/paper/2604.10079/) (2026)
- [RECAP: REwriting Conversations for Intent Understanding in Agentic Planning](https://mlsanity.com/paper/2509.04472/) (2025)
- [A Survey of Ontology Expansion for Conversational Understanding](https://mlsanity.com/paper/2410.15019/) (2024)
- [Multi-Perspective Consistency Enhances Confidence Estimation in Large Language Models](https://mlsanity.com/paper/2402.11279/) (2024)
- [Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulation](https://mlsanity.com/paper/2505.18556/) (2025)
- [LANID: LLM-assisted New Intent Discovery](https://mlsanity.com/paper/2503.23740/) (2025)
- [Noise-BERT: A Unified Perturbation-Robust Framework with Noise Alignment Pre-training for Noisy Slot Filling Task](https://mlsanity.com/paper/2402.14494/) (2024)
- [A Survey on the Real Power of ChatGPT](https://mlsanity.com/paper/2405.00704/) (2024)
- [Pseudo-Label Enhanced Prototypical Contrastive Learning for Uniformed Intent Discovery](https://mlsanity.com/paper/2410.20219/) (2024)
- [A Computational Approach to Modeling Conversational Systems: Analyzing Large-Scale Quasi-Patterned Dialogue Flows](https://mlsanity.com/paper/2507.13544/) (2025)
- [Toward Machine Interpreting: Lessons from Human Interpreting Studies](https://mlsanity.com/paper/2508.07964/) (2025)
- [NILC: Discovering New Intents with LLM-assisted Clustering](https://mlsanity.com/paper/2511.05913/) (2025)

## Abstract (arXiv)

The tasks of out-of-domain (OOD) intent discovery and generalized intent discovery (GID) aim to extend a closed intent classifier to open-world intent sets, which is crucial to task-oriented dialogue (TOD) systems. Previous methods address them by fine-tuning discriminative models. Recently, although some studies have been exploring the application of large language models (LLMs) represented by ChatGPT to various downstream tasks, it is still unclear for the ability of ChatGPT to discover and incrementally extent OOD intents. In this paper, we comprehensively evaluate ChatGPT on OOD intent discovery and GID, and then outline the strengths and weaknesses of ChatGPT. Overall, ChatGPT exhibits consistent advantages under zero-shot settings, but is still at a disadvantage compared to fine-tuned models. More deeply, through a series of analytical experiments, we summarize and discuss the challenges faced by LLMs including clustering, domain-specific understanding, and cross-domain in-context learning scenarios. Finally, we provide empirical guidance for future directions to address these challenges.