# Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference

arXiv: https://arxiv.org/abs/2010.13009 · HTML: https://mlsanity.com/paper/2010.13009/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Oct 2020.

## Built on by (most cited first, 30 of 47)

- [PTR: Prompt Tuning with Rules for Text Classification](https://mlsanity.com/paper/2105.11259/) (2021)
- [Neural Data Augmentation via Example Extrapolation](https://mlsanity.com/paper/2102.01335/) (2021)
- [Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning](https://mlsanity.com/paper/2109.06349/) (2021)
- [Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training](https://mlsanity.com/paper/2106.08616/) (2021)
- [Unsupervised Out-of-Domain Detection via Pre-trained Transformers](https://mlsanity.com/paper/2106.00948/) (2021)
- [Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System](https://mlsanity.com/paper/2104.11882/) (2021)
- [New Intent Discovery with Pre-training and Contrastive Learning](https://mlsanity.com/paper/2205.12914/) (2022)
- [SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for Task-Oriented Dialog Understanding](https://mlsanity.com/paper/2209.06638/) (2022)
- [Effectiveness of Pre-training for Few-shot Intent Classification](https://mlsanity.com/paper/2109.05782/) (2021)
- [ConvFiT: Conversational Fine-Tuning of Pretrained Language Models](https://mlsanity.com/paper/2109.10126/) (2021)
- [Selective In-Context Data Augmentation for Intent Detection using Pointwise V-Information](https://mlsanity.com/paper/2302.05096/) (2023)
- [Are Pretrained Transformers Robust in Intent Classification? A Missing Ingredient in Evaluation of Out-of-Scope Intent Detection](https://mlsanity.com/paper/2106.04564/) (2021)
- [Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropization](https://mlsanity.com/paper/2205.07208/) (2022)
- [Label Semantic Aware Pre-training for Few-shot Text Classification](https://mlsanity.com/paper/2204.07128/) (2022)
- [Learning Discriminative Representations and Decision Boundaries for Open Intent Detection](https://mlsanity.com/paper/2203.05823/) (2022)
- [Towards Textual Out-of-Domain Detection without In-Domain Labels](https://mlsanity.com/paper/2203.11396/) (2022)
- [Pseudo Siamese Network for Few-shot Intent Generation](https://mlsanity.com/paper/2105.00896/) (2021)
- [IntentGPT: Few-shot Intent Discovery with Large Language Models](https://mlsanity.com/paper/2411.10670/) (2024)
- [Converse: A Tree-Based Modular Task-Oriented Dialogue System](https://mlsanity.com/paper/2203.12187/) (2022)
- [Generate then Refine: Data Augmentation for Zero-shot Intent Detection](https://mlsanity.com/paper/2410.01953/) (2024)
- [Zero-Shot Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2202.01924/) (2022)
- [Learning to Classify Open Intent via Soft Labeling and Manifold Mixup](https://mlsanity.com/paper/2204.07804/) (2022)
- [QAID: Question Answering Inspired Few-shot Intent Detection](https://mlsanity.com/paper/2303.01593/) (2023)
- [Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning](https://mlsanity.com/paper/2304.01295/) (2023)
- [Revisit Few-shot Intent Classification with PLMs: Direct Fine-tuning vs. Continual Pre-training](https://mlsanity.com/paper/2306.05278/) (2023)
- [InstructTODS: Large Language Models for End-to-End Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2310.08885/) (2023)
- [ProtoInfoMax: Prototypical Networks with Mutual Information Maximization for Out-of-Domain Detection](https://mlsanity.com/paper/2108.12229/) (2021)
- [Learning to Select from Multiple Options](https://mlsanity.com/paper/2212.00301/) (2022)
- [Domain Adaptation in Intent Classification Systems: A Review](https://mlsanity.com/paper/2404.14415/) (2024)
- [InsightNet: Structured Insight Mining from Customer Feedback](https://mlsanity.com/paper/2405.07195/) (2024)

## Abstract (arXiv)

Intent detection is one of the core components of goal-oriented dialog systems, and detecting out-of-scope (OOS) intents is also a practically important skill. Few-shot learning is attracting much attention to mitigate data scarcity, but OOS detection becomes even more challenging. In this paper, we present a simple yet effective approach, discriminative nearest neighbor classification with deep self-attention. Unlike softmax classifiers, we leverage BERT-style pairwise encoding to train a binary classifier that estimates the best matched training example for a user input. We propose to boost the discriminative ability by transferring a natural language inference (NLI) model. Our extensive experiments on a large-scale multi-domain intent detection task show that our method achieves more stable and accurate in-domain and OOS detection accuracy than RoBERTa-based classifiers and embedding-based nearest neighbor approaches. More notably, the NLI transfer enables our 10-shot model to perform competitively with 50-shot or even full-shot classifiers, while we can keep the inference time constant by leveraging a faster embedding retrieval model.