mlsanityData to 7 Oct 2026

Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection

arXiv 2402.17256 ↗First posted 27 Feb 2024cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

Out-of-domain (OOD) intent detection aims to examine whether the user's query falls outside the predefined domain of the system, which is crucial for the proper functioning of task-oriented dialogue (TOD) systems. Previous methods address it by fine-tuning discriminative models. Recently, some studies have been exploring the application of large language models (LLMs) represented by ChatGPT to various downstream tasks, but it is still unclear for their ability on OOD detection task.This paper conducts a comprehensive evaluation of LLMs under various experimental settings, and then outline the strengths and weaknesses of LLMs. We find that LLMs exhibit strong zero-shot and few-shot capabilities, but is still at a disadvantage compared to models fine-tuned with full resource. More deeply, through a series of additional analysis experiments, we discuss and summarize the challenges faced by LLMs and provide guidance for future work including injecting domain knowledge, strengthening knowledge transfer from IND(In-domain) to OOD, and understanding long instructions.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 26 of its 39 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

26 of 39 references.

References · 26

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  4. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  5. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  6. 2015Towards Open Set Deep Networkscited by 533
  7. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  8. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 419
  9. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 378
  10. 2023ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPTcited by 250
  11. 2019BERT for Joint Intent Classification and Slot Fillingcited by 195
  12. 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 172
14 more
  1. 2023Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarizationcited by 122
  2. 2017DOC: Deep Open Classification of Text Documentscited by 109
  3. 2019Deep Unknown Intent Detection with Margin Losscited by 66
  4. 2019Out-of-domain Detection for Natural Language Understanding in Dialog Systemscited by 62
  5. 2021Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learningcited by 42
  6. 2023A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understandingcited by 31
  7. 2018Joint Learning of Domain Classification and Out-of-Domain Detection with Dynamic Class Weighting for Satisficing False Acceptance Ratescited by 17
  8. 2023Large Language Models Meet Open-World Intent Discovery and Recognition: An Evaluation of ChatGPTcited by 16
  9. 2022Distribution Calibration for Out-of-Domain Detection with Bayesian Approximationcited by 7
  10. 2022UniNL: Aligning Representation Learning with Scoring Function for OOD Detection via Unified Neighborhood Learningcited by 5
  11. 2023Decoupling Pseudo Label Disambiguation and Representation Learning for Generalized Intent Discoverycited by 5
  12. 2023Continual Generalized Intent Discovery: Marching Towards Dynamic and Open-world Intent Recognitioncited by 5
  13. 2023APP: Adaptive Prototypical Pseudo-Labeling for Few-shot OOD Detectioncited by 3
  14. 2023Probing Out-of-Distribution Robustness of Language Models with Parameter-Efficient Transfer Learningcited by 2

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 10.

14 papers.

Sort:

By area:

  1. 2024When Search Engine Services meet Large Language Models: Visions and Challengescited by 59
  2. 2024CS-Bench: A Comprehensive Benchmark for Large Language Models towards Computer Science Masterycited by 30
  3. 2024Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advancescited by 24
  4. 2025Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Modelscited by 14
  5. 2025RECAP: REwriting Conversations for Intent Understanding in Agentic Planningcited by 7
  6. 2025Toward Super Agent System with Hybrid AI Routerscited by 6
  7. 2025Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulationcited by 3
  8. 2024Knowledge Graphs are all you need: Leveraging KGs in Physics Question Answeringcited by 2
  9. 2025Intent Recognition and Out-of-Scope Detection using LLMs in Multi-party Conversationscited by 2
  10. 2025MiST: Understanding the Role of Mid-Stage Scientific Training in Developing Chemical Reasoning Modelscited by 2
  11. 2025Efficient Out-of-Scope Detection in Dialogue Systems via Uncertainty-Driven LLM Routingcited by 1
  12. 2025Large Language Model Data Generation for Enhanced Intent Recognition in German Speechcited by 1
2 more
  1. 2025A Comparative Study of Task Adaptation Techniques of Large Language Models for Identifying Sustainable Development Goalscited by 0
  2. 2025DROID: Dual Representation for Out-of-Scope Intent Detectioncited by 0