# Conformal Intent Classification and Clarification for Fast and Accurate Intent Recognition

arXiv: https://arxiv.org/abs/2403.18973 · HTML: https://mlsanity.com/paper/2403.18973/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 43 references that are in the map.
- First posted on arXiv 27 Mar 2024.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Uncertainty Sets for Image Classifiers using Conformal Prediction](https://mlsanity.com/paper/2009.14193/) (2020)
- [Pretrained Transformers Improve Out-of-Distribution Robustness](https://mlsanity.com/paper/2004.06100/) (2020)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Benchmarking Natural Language Understanding Services for building Conversational Agents](https://mlsanity.com/paper/1903.05566/) (2019)
- [Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialog](https://mlsanity.com/paper/1810.13327/) (2018)
- [DOC: Deep Open Classification of Text Documents](https://mlsanity.com/paper/1709.08716/) (2017)
- [Contrastive Out-of-Distribution Detection for Pretrained Transformers](https://mlsanity.com/paper/2104.08812/) (2021)
- [Deep Unknown Intent Detection with Margin Loss](https://mlsanity.com/paper/1906.00434/) (2019)
- [Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training](https://mlsanity.com/paper/2106.08616/) (2021)
- [Conformal Bayesian Computation](https://mlsanity.com/paper/2106.06137/) (2021)
- [Conformal Prediction Sets with Limited False Positives](https://mlsanity.com/paper/2202.07650/) (2022)
- [Resolving Intent Ambiguities by Retrieving Discriminative Clarifying Questions](https://mlsanity.com/paper/2008.07559/) (2020)
- [Diverse and Specific Clarification Question Generation with Keywords](https://mlsanity.com/paper/2104.10317/) (2021)
- [Reinforcement Learning for Personalized Dialogue Management](https://mlsanity.com/paper/1908.00286/) (2019)

## Built on by (most cited first, 5 of 5)

- [Dial-In LLM: Human-Aligned LLM-in-the-loop Intent Clustering for Customer Service Dialogues](https://mlsanity.com/paper/2412.09049/) (2024)
- [Hierarchical Conformal Classification](https://mlsanity.com/paper/2508.13288/) (2025)
- [Detecting Linguistic Bias in Government Documents Using Large language Models](https://mlsanity.com/paper/2502.13548/) (2025)
- [The Significance of Style Diversity in Annotation-Free Synthetic Data Generation](https://mlsanity.com/paper/2606.20400/) (2026)
- [Lost in the Fallback: How Reliably Do Typed Decision Models Use Fallback Options?](https://mlsanity.com/paper/2610.19856/) (2026)

## Abstract (arXiv)

We present Conformal Intent Classification and Clarification (CICC), a framework for fast and accurate intent classification for task-oriented dialogue systems. The framework turns heuristic uncertainty scores of any intent classifier into a clarification question that is guaranteed to contain the true intent at a pre-defined confidence level. By disambiguating between a small number of likely intents, the user query can be resolved quickly and accurately. Additionally, we propose to augment the framework for out-of-scope detection. In a comparative evaluation using seven intent recognition datasets we find that CICC generates small clarification questions and is capable of out-of-scope detection. CICC can help practitioners and researchers substantially in improving the user experience of dialogue agents with specific clarification questions.