# A Simple Meta-learning Paradigm for Zero-shot Intent Classification with Mixture Attention Mechanism

arXiv: https://arxiv.org/abs/2206.02179 · HTML: https://mlsanity.com/paper/2206.02179/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 44 references that are in the map.
- First posted on arXiv 5 Jun 2022.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [Zero-Shot Learning Through Cross-Modal Transfer](https://mlsanity.com/paper/1301.3666/) (2013)
- [A Survey on Dialogue Systems: Recent Advances and New Frontiers](https://mlsanity.com/paper/1711.01731/) (2017)
- [A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding](https://mlsanity.com/paper/1909.02188/) (2019)
- [DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding](https://mlsanity.com/paper/1709.04696/) (2017)
- [Episode-based Prototype Generating Network for Zero-Shot Learning](https://mlsanity.com/paper/1909.03360/) (2019)
- [A Co-Interactive Transformer for Joint Slot Filling and Intent Detection](https://mlsanity.com/paper/2010.03880/) (2020)
- [Few-shot Text Classification with Distributional Signatures](https://mlsanity.com/paper/1908.06039/) (2019)
- [Zero-shot User Intent Detection via Capsule Neural Networks](https://mlsanity.com/paper/1809.00385/) (2018)
- [Analogical Reasoning on Chinese Morphological and Semantic Relations](https://mlsanity.com/paper/1805.06504/) (2018)
- [Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training](https://mlsanity.com/paper/2106.08616/) (2021)
- [Few-shot Intent Classification and Slot Filling with Retrieved Examples](https://mlsanity.com/paper/2104.05763/) (2021)
- [Integrating Semantic Knowledge to Tackle Zero-shot Text Classification](https://mlsanity.com/paper/1903.12626/) (2019)
- [Enhancing the Generalization for Intent Classification and Out-of-Domain Detection in SLU](https://mlsanity.com/paper/2106.14464/) (2021)
- [Generalized Zero-shot Intent Detection via Commonsense Knowledge](https://mlsanity.com/paper/2102.02925/) (2021)
- [Meta-Learning Adversarial Domain Adaptation Network for Few-Shot Text Classification](https://mlsanity.com/paper/2107.12262/) (2021)
- [Pseudo Siamese Network for Few-shot Intent Generation](https://mlsanity.com/paper/2105.00896/) (2021)
- [An Explicit-Joint and Supervised-Contrastive Learning Framework for Few-Shot Intent Classification and Slot Filling](https://mlsanity.com/paper/2110.13691/) (2021)
- [ProtoDA: Efficient Transfer Learning for Few-Shot Intent Classification](https://mlsanity.com/paper/2101.11753/) (2021)
- [The First Evaluation of Chinese Human-Computer Dialogue Technology](https://mlsanity.com/paper/1709.10217/) (2017)
- [Learning Class-Transductive Intent Representations for Zero-shot Intent Detection](https://mlsanity.com/paper/2012.01721/) (2020)

## Built on by (most cited first, 6 of 6)

- [Why Misinformation is Created? Detecting them by Integrating Intent Features](https://mlsanity.com/paper/2407.19196/) (2024)
- [A Survey of Ontology Expansion for Conversational Understanding](https://mlsanity.com/paper/2410.15019/) (2024)
- [Pre-training Intent-Aware Encoders for Zero- and Few-Shot Intent Classification](https://mlsanity.com/paper/2305.14827/) (2023)
- [Z-BERT-A: a zero-shot Pipeline for Unknown Intent detection](https://mlsanity.com/paper/2208.07084/) (2022)
- [Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systems](https://mlsanity.com/paper/2307.07255/) (2023)
- [Exploring Description-Augmented Dataless Intent Classification](https://mlsanity.com/paper/2407.17862/) (2024)

## Abstract (arXiv)

Zero-shot intent classification is a vital and challenging task in dialogue systems, which aims to deal with numerous fast-emerging unacquainted intents without annotated training data. To obtain more satisfactory performance, the crucial points lie in two aspects: extracting better utterance features and strengthening the model generalization ability. In this paper, we propose a simple yet effective meta-learning paradigm for zero-shot intent classification. To learn better semantic representations for utterances, we introduce a new mixture attention mechanism, which encodes the pertinent word occurrence patterns by leveraging the distributional signature attention and multi-layer perceptron attention simultaneously. To strengthen the transfer ability of the model from seen classes to unseen classes, we reformulate zero-shot intent classification with a meta-learning strategy, which trains the model by simulating multiple zero-shot classification tasks on seen categories, and promotes the model generalization ability with a meta-adapting procedure on mimic unseen categories. Extensive experiments on two real-world dialogue datasets in different languages show that our model outperforms other strong baselines on both standard and generalized zero-shot intent classification tasks.