# Few-shot Intent Classification and Slot Filling with Retrieved Examples

arXiv: https://arxiv.org/abs/2104.05763 · HTML: https://mlsanity.com/paper/2104.05763/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 44 references that are in the map.
- First posted on arXiv 12 Apr 2021.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples](https://mlsanity.com/paper/1903.03096/) (2019)
- [Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1905.08743/) (2019)
- [SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning](https://mlsanity.com/paper/1911.04623/) (2019)
- [Learning Dense Representations for Entity Retrieval](https://mlsanity.com/paper/1909.10506/) (2019)
- [Revisiting Training Strategies and Generalization Performance in Deep Metric Learning](https://mlsanity.com/paper/2002.08473/) (2020)
- [TapNet: Neural Network Augmented with Task-Adaptive Projection for Few-Shot Learning](https://mlsanity.com/paper/1905.06549/) (2019)
- [Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network](https://mlsanity.com/paper/2006.05702/) (2020)
- [Few-shot classification in Named Entity Recognition Task](https://mlsanity.com/paper/1812.06158/) (2018)
- [Induction Networks for Few-Shot Text Classification](https://mlsanity.com/paper/1902.10482/) (2019)
- [Guiding Neural Machine Translation with Retrieved Translation Pieces](https://mlsanity.com/paper/1804.02559/) (2018)
- [Diverse Few-Shot Text Classification with Multiple Metrics](https://mlsanity.com/paper/1805.07513/) (2018)
- [Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmax](https://mlsanity.com/paper/1902.08564/) (2019)
- [Learning to Classify Intents and Slot Labels Given a Handful of Examples](https://mlsanity.com/paper/2004.10793/) (2020)
- [Text Generation with Exemplar-based Adaptive Decoding](https://mlsanity.com/paper/1904.04428/) (2019)
- [Label-Agnostic Sequence Labeling by Copying Nearest Neighbors](https://mlsanity.com/paper/1906.04225/) (2019)
- [A Cross-Task Analysis of Text Span Representations](https://mlsanity.com/paper/2006.03866/) (2020)
- [MIDAS: A Dialog Act Annotation Scheme for Open Domain Human Machine Spoken Conversations](https://mlsanity.com/paper/1908.10023/) (2019)

## Built on by (most cited first, 30 of 35)

- [In-Context Learning for Few-Shot Dialogue State Tracking](https://mlsanity.com/paper/2203.08568/) (2022)
- [Controllable Dialogue Simulation with In-Context Learning](https://mlsanity.com/paper/2210.04185/) (2022)
- [SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Prompting](https://mlsanity.com/paper/2305.09067/) (2023)
- [Controllable Semantic Parsing via Retrieval Augmentation](https://mlsanity.com/paper/2110.08458/) (2021)
- [An Enhanced Span-based Decomposition Method for Few-Shot Sequence Labeling](https://mlsanity.com/paper/2109.13023/) (2021)
- [Selective In-Context Data Augmentation for Intent Detection using Pointwise V-Information](https://mlsanity.com/paper/2302.05096/) (2023)
- [Decomposed Meta-Learning for Few-Shot Named Entity Recognition](https://mlsanity.com/paper/2204.05751/) (2022)
- [Exploring Zero and Few-shot Techniques for Intent Classification](https://mlsanity.com/paper/2305.07157/) (2023)
- [Label Semantic Aware Pre-training for Few-shot Text Classification](https://mlsanity.com/paper/2204.07128/) (2022)
- [Learning Dialogue Representations from Consecutive Utterances](https://mlsanity.com/paper/2205.13568/) (2022)
- [Unsupervised Slot Schema Induction for Task-oriented Dialog](https://mlsanity.com/paper/2205.04515/) (2022)
- [LARA: Linguistic-Adaptive Retrieval-Augmentation for Multi-Turn Intent Classification](https://mlsanity.com/paper/2403.16504/) (2024)
- [An Explicit-Joint and Supervised-Contrastive Learning Framework for Few-Shot Intent Classification and Slot Filling](https://mlsanity.com/paper/2110.13691/) (2021)
- [Label-enhanced Prototypical Network with Contrastive Learning for Multi-label Few-shot Aspect Category Detection](https://mlsanity.com/paper/2206.13980/) (2022)
- [A Simple Meta-learning Paradigm for Zero-shot Intent Classification with Mixture Attention Mechanism](https://mlsanity.com/paper/2206.02179/) (2022)
- [Measuring and Mitigating Constraint Violations of In-Context Learning for Utterance-to-API Semantic Parsing](https://mlsanity.com/paper/2305.15338/) (2023)
- [Inverse is Better! Fast and Accurate Prompt for Few-shot Slot Tagging](https://mlsanity.com/paper/2204.00885/) (2022)
- [Knowledge-grounded Dialog State Tracking](https://mlsanity.com/paper/2210.06656/) (2022)
- [PromptNER: A Prompting Method for Few-shot Named Entity Recognition via k Nearest Neighbor Search](https://mlsanity.com/paper/2305.12217/) (2023)
- [Revisit Few-shot Intent Classification with PLMs: Direct Fine-tuning vs. Continual Pre-training](https://mlsanity.com/paper/2306.05278/) (2023)
- [Robust Few-Shot Named Entity Recognition with Boundary Discrimination and Correlation Purification](https://mlsanity.com/paper/2312.07961/) (2023)
- [Controllable and Diverse Data Augmentation with Large Language Model for Low-Resource Open-Domain Dialogue Generation](https://mlsanity.com/paper/2404.00361/) (2024)
- [REIC: RAG-Enhanced Intent Classification at Scale](https://mlsanity.com/paper/2506.00210/) (2025)
- [Towards Zero-Shot Frame Semantic Parsing with Task Agnostic Ontologies and Simple Labels](https://mlsanity.com/paper/2305.03793/) (2023)
- [Few-Shot NLU with Vector Projection Distance and Abstract Triangular CRF](https://mlsanity.com/paper/2112.04999/) (2021)
- [A Low-Cost, Controllable and Interpretable Task-Oriented Chatbot: With Real-World After-Sale Services as Example](https://mlsanity.com/paper/2205.06436/) (2022)
- [Low-Resource Compositional Semantic Parsing with Concept Pretraining](https://mlsanity.com/paper/2301.09809/) (2023)
- [Multi-Stage Coarse-to-Fine Contrastive Learning for Conversation Intent Induction](https://mlsanity.com/paper/2303.05034/) (2023)
- [Dialog-to-Actions: Building Task-Oriented Dialogue System via Action-Level Generation](https://mlsanity.com/paper/2304.00884/) (2023)
- [Template-assisted Contrastive Learning of Task-oriented Dialogue Sentence Embeddings](https://mlsanity.com/paper/2305.14299/) (2023)

## Abstract (arXiv)

Few-shot learning arises in important practical scenarios, such as when a natural language understanding system needs to learn new semantic labels for an emerging, resource-scarce domain. In this paper, we explore retrieval-based methods for intent classification and slot filling tasks in few-shot settings. Retrieval-based methods make predictions based on labeled examples in the retrieval index that are similar to the input, and thus can adapt to new domains simply by changing the index without having to retrain the model. However, it is non-trivial to apply such methods on tasks with a complex label space like slot filling. To this end, we propose a span-level retrieval method that learns similar contextualized representations for spans with the same label via a novel batch-softmax objective. At inference time, we use the labels of the retrieved spans to construct the final structure with the highest aggregated score. Our method outperforms previous systems in various few-shot settings on the CLINC and SNIPS benchmarks.