# Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection

arXiv: https://arxiv.org/abs/2402.17256 · HTML: https://mlsanity.com/paper/2402.17256/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 27 Feb 2024.

## Builds on (most cited first, 26 of 26)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Likelihood Ratios for Out-of-Distribution Detection](https://mlsanity.com/paper/1906.02845/) (2019)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine](https://mlsanity.com/paper/2301.08745/) (2023)
- [Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarization](https://mlsanity.com/paper/2302.08081/) (2023)
- [DOC: Deep Open Classification of Text Documents](https://mlsanity.com/paper/1709.08716/) (2017)
- [Deep Unknown Intent Detection with Margin Loss](https://mlsanity.com/paper/1906.00434/) (2019)
- [Out-of-domain Detection for Natural Language Understanding in Dialog Systems](https://mlsanity.com/paper/1909.03862/) (2019)
- [Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning](https://mlsanity.com/paper/2105.14289/) (2021)
- [A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding](https://mlsanity.com/paper/2304.04256/) (2023)
- [Joint Learning of Domain Classification and Out-of-Domain Detection with Dynamic Class Weighting for Satisficing False Acceptance Rates](https://mlsanity.com/paper/1807.00072/) (2018)
- [Large Language Models Meet Open-World Intent Discovery and Recognition: An Evaluation of ChatGPT](https://mlsanity.com/paper/2310.10176/) (2023)
- [Distribution Calibration for Out-of-Domain Detection with Bayesian Approximation](https://mlsanity.com/paper/2209.06612/) (2022)
- [UniNL: Aligning Representation Learning with Scoring Function for OOD Detection via Unified Neighborhood Learning](https://mlsanity.com/paper/2210.10722/) (2022)
- [Decoupling Pseudo Label Disambiguation and Representation Learning for Generalized Intent Discovery](https://mlsanity.com/paper/2305.17699/) (2023)
- [Continual Generalized Intent Discovery: Marching Towards Dynamic and Open-world Intent Recognition](https://mlsanity.com/paper/2310.10184/) (2023)
- [APP: Adaptive Prototypical Pseudo-Labeling for Few-shot OOD Detection](https://mlsanity.com/paper/2310.13380/) (2023)
- [Probing Out-of-Distribution Robustness of Language Models with Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2301.11660/) (2023)

## Built on by (most cited first, 14 of 14)

- [When Search Engine Services meet Large Language Models: Visions and Challenges](https://mlsanity.com/paper/2407.00128/) (2024)
- [CS-Bench: A Comprehensive Benchmark for Large Language Models towards Computer Science Mastery](https://mlsanity.com/paper/2406.08587/) (2024)
- [Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advances](https://mlsanity.com/paper/2409.11884/) (2024)
- [Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models](https://mlsanity.com/paper/2501.05382/) (2025)
- [RECAP: REwriting Conversations for Intent Understanding in Agentic Planning](https://mlsanity.com/paper/2509.04472/) (2025)
- [Toward Super Agent System with Hybrid AI Routers](https://mlsanity.com/paper/2504.10519/) (2025)
- [Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulation](https://mlsanity.com/paper/2505.18556/) (2025)
- [MiST: Understanding the Role of Mid-Stage Scientific Training in Developing Chemical Reasoning Models](https://mlsanity.com/paper/2512.21231/) (2025)
- [Knowledge Graphs are all you need: Leveraging KGs in Physics Question Answering](https://mlsanity.com/paper/2412.05453/) (2024)
- [Intent Recognition and Out-of-Scope Detection using LLMs in Multi-party Conversations](https://mlsanity.com/paper/2507.22289/) (2025)
- [Efficient Out-of-Scope Detection in Dialogue Systems via Uncertainty-Driven LLM Routing](https://mlsanity.com/paper/2507.01541/) (2025)
- [Large Language Model Data Generation for Enhanced Intent Recognition in German Speech](https://mlsanity.com/paper/2508.06277/) (2025)
- [A Comparative Study of Task Adaptation Techniques of Large Language Models for Identifying Sustainable Development Goals](https://mlsanity.com/paper/2506.15208/) (2025)
- [DROID: Dual Representation for Out-of-Scope Intent Detection](https://mlsanity.com/paper/2510.14110/) (2025)

## Abstract (arXiv)

Out-of-domain (OOD) intent detection aims to examine whether the user's query falls outside the predefined domain of the system, which is crucial for the proper functioning of task-oriented dialogue (TOD) systems. Previous methods address it by fine-tuning discriminative models. Recently, some studies have been exploring the application of large language models (LLMs) represented by ChatGPT to various downstream tasks, but it is still unclear for their ability on OOD detection task.This paper conducts a comprehensive evaluation of LLMs under various experimental settings, and then outline the strengths and weaknesses of LLMs. We find that LLMs exhibit strong zero-shot and few-shot capabilities, but is still at a disadvantage compared to models fine-tuned with full resource. More deeply, through a series of additional analysis experiments, we discuss and summarize the challenges faced by LLMs and provide guidance for future work including injecting domain knowledge, strengthening knowledge transfer from IND(In-domain) to OOD, and understanding long instructions.