mlsanityData to 7 Oct 2026

Large Language Models Meet Open-World Intent Discovery and Recognition: An Evaluation of ChatGPT

arXiv 2310.10176 ↗First posted 16 Oct 2023cs.CL cs.AI cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

The tasks of out-of-domain (OOD) intent discovery and generalized intent discovery (GID) aim to extend a closed intent classifier to open-world intent sets, which is crucial to task-oriented dialogue (TOD) systems. Previous methods address them by fine-tuning discriminative models. Recently, although some studies have been exploring the application of large language models (LLMs) represented by ChatGPT to various downstream tasks, it is still unclear for the ability of ChatGPT to discover and incrementally extent OOD intents. In this paper, we comprehensively evaluate ChatGPT on OOD intent discovery and GID, and then outline the strengths and weaknesses of ChatGPT. Overall, ChatGPT exhibits consistent advantages under zero-shot settings, but is still at a disadvantage compared to fine-tuned models. More deeply, through a series of analytical experiments, we summarize and discuss the challenges faced by LLMs including clustering, domain-specific understanding, and cross-domain in-context learning scenarios. Finally, we provide empirical guidance for future directions to address these challenges.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 25 of its 35 references that are in the map. First posted on arXiv 16 Oct 2023.

Builds on

25 of 35 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  6. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  7. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  8. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  9. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 419
  10. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 378
  11. 2023Mathematical Capabilities of ChatGPTcited by 263
  12. 2023On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspectivecited by 208
13 more
  1. 2019BERT for Joint Intent Classification and Slot Fillingcited by 195
  2. 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 172
  3. 2023Unlimiformer: Long-Range Transformers with Unlimited Length Inputcited by 127
  4. 2020Discovering New Intents with Deep Aligned Clusteringcited by 68
  5. 2019Discovering New Intents via Constrained Deep Adaptive Clustering with Cluster Refinementcited by 56
  6. 2023On the Educational Impact of ChatGPT: Is Artificial Intelligence Ready to Obtain a University Degree?cited by 47
  7. 2023Revisit Input Perturbation Problems for LLMs: A Unified Robustness Evaluation Framework for Noisy Slot Filling Taskcited by 23
  8. 2022Semi-Supervised Knowledge-Grounded Pre-training for Task-Oriented Dialog Systemscited by 15
  9. 2022Generalized Intent Discovery: Learning from Open World Dialogue Systemcited by 10
  10. 2022PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Fillingcited by 9
  11. 2022Watch the Neighbors: A Unified K-Nearest Neighbor Contrastive Learning Framework for OOD Intent Discoverycited by 9
  12. 2023Revisit Out-Of-Vocabulary Problem for Slot Filling: A Unified Contrastive Frameword with Multi-level Data Augmentationscited by 8
  13. 2023Towards Robust and Generalizable Training: An Empirical Study of Noisy Slot Filling for Input Perturbationscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 7; 2026: 1.

16 papers.

Sort:

By area:

  1. 2025Thought-Augmented Planning for LLM-Powered Interactive Recommender Agentcited by 17
  2. 2024Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detectioncited by 14
  3. 2024DolphCoder: Echo-Locating Code Large Language Models with Diverse and Multi-Objective Instruction Tuningcited by 11
  4. 2024Dial-In LLM: Human-Aligned LLM-in-the-loop Intent Clustering for Customer Service Dialoguescited by 11
  5. 2025RECAP: REwriting Conversations for Intent Understanding in Agentic Planningcited by 7
  6. 2026Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Modelscited by 7
  7. 2024A Survey of Ontology Expansion for Conversational Understandingcited by 6
  8. 2024Multi-Perspective Consistency Enhances Confidence Estimation in Large Language Modelscited by 4
  9. 2025LANID: LLM-assisted New Intent Discoverycited by 3
  10. 2025Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulationcited by 3
  11. 2024Noise-BERT: A Unified Perturbation-Robust Framework with Noise Alignment Pre-training for Noisy Slot Filling Taskcited by 1
  12. 2024A Survey on the Real Power of ChatGPTcited by 0
4 more
  1. 2024Pseudo-Label Enhanced Prototypical Contrastive Learning for Uniformed Intent Discoverycited by 0
  2. 2025A Computational Approach to Modeling Conversational Systems: Analyzing Large-Scale Quasi-Patterned Dialogue Flowscited by 0
  3. 2025Toward Machine Interpreting: Lessons from Human Interpreting Studiescited by 0
  4. 2025NILC: Discovering New Intents with LLM-assisted Clusteringcited by 0