# Few-shot Learning for Slot Tagging with Attentive Relational Network

arXiv: https://arxiv.org/abs/2103.02333 · HTML: https://mlsanity.com/paper/2103.02333/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 37 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [Advances in Pre-Training Distributed Word Representations](https://mlsanity.com/paper/1712.09405/) (2017)
- [Meta-Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1808.08437/) (2018)
- [TapNet: Neural Network Augmented with Task-Adaptive Projection for Few-Shot Learning](https://mlsanity.com/paper/1905.06549/) (2019)
- [Learn To Pay Attention](https://mlsanity.com/paper/1804.02391/) (2018)
- [Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network](https://mlsanity.com/paper/2006.05702/) (2020)
- [Induction Networks for Few-Shot Text Classification](https://mlsanity.com/paper/1902.10482/) (2019)
- [Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Tasks](https://mlsanity.com/paper/1908.10423/) (2019)
- [Towards Zero-Shot Frame Semantic Parsing for Domain Scaling](https://mlsanity.com/paper/1707.02363/) (2017)
- [Robust Zero-Shot Cross-Domain Slot Filling with Example Values](https://mlsanity.com/paper/1906.06870/) (2019)
- [Leveraging Sentence-level Information with Encoder LSTM for Semantic Slot Filling](https://mlsanity.com/paper/1601.01530/) (2016)
- [Zero-Shot Adaptive Transfer for Conversational Language Understanding](https://mlsanity.com/paper/1808.10059/) (2018)
- [Neural Models for Sequence Chunking](https://mlsanity.com/paper/1701.04027/) (2017)
- [Encoder-decoder with Focus-mechanism for Sequence Labelling Based Spoken Language Understanding](https://mlsanity.com/paper/1608.02097/) (2016)
- [Recurrent Neural Networks with External Memory for Language Understanding](https://mlsanity.com/paper/1506.00195/) (2015)
- [Domain Adaptation of Recurrent Neural Networks for Natural Language Understanding](https://mlsanity.com/paper/1604.00117/) (2016)
- [Concept Transfer Learning for Adaptive Language Understanding](https://mlsanity.com/paper/1706.00927/) (2017)

## Built on by (most cited first, 6 of 6)

- [Meta Learning for Natural Language Processing: A Survey](https://mlsanity.com/paper/2205.01500/) (2022)
- [A Survey of Ontology Expansion for Conversational Understanding](https://mlsanity.com/paper/2410.15019/) (2024)
- [Improving Few-shot Generalization of Safety Classifiers via Data Augmented Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2310.16959/) (2023)
- [Few-shot Named Entity Recognition with Cloze Questions](https://mlsanity.com/paper/2111.12421/) (2021)
- [Unifying Token and Span Level Supervisions for Few-Shot Sequence Labeling](https://mlsanity.com/paper/2307.07946/) (2023)
- [Actively Discovering New Slots for Task-oriented Conversation](https://mlsanity.com/paper/2305.04049/) (2023)

## Abstract (arXiv)

Metric-based learning is a well-known family of methods for few-shot learning, especially in computer vision. Recently, they have been used in many natural language processing applications but not for slot tagging. In this paper, we explore metric-based learning methods in the slot tagging task and propose a novel metric-based learning architecture - Attentive Relational Network. Our proposed method extends relation networks, making them more suitable for natural language processing applications in general, by leveraging pretrained contextual embeddings such as ELMO and BERT and by using attention mechanism. The results on SNIPS data show that our proposed method outperforms other state-of-the-art metric-based learning methods.