# Joint Multiple Intent Detection and Slot Filling with Supervised Contrastive Learning and Self-Distillation

arXiv: https://arxiv.org/abs/2308.14654 · HTML: https://mlsanity.com/paper/2308.14654/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation.
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 28 Aug 2023.

## Placed in

- Knowledge distillation / [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/)

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://mlsanity.com/paper/2011.01403/) (2020)
- [Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1609.01454/) (2016)
- [A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding](https://mlsanity.com/paper/1909.02188/) (2019)
- [Self-Guided Contrastive Learning for BERT Sentence Representations](https://mlsanity.com/paper/2106.07345/) (2021)
- [A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1907.00390/) (2019)
- [A Co-Interactive Transformer for Joint Slot Filling and Intent Detection](https://mlsanity.com/paper/2010.03880/) (2020)
- [Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework](https://mlsanity.com/paper/2204.13207/) (2022)
- [A Bi-model based RNN Semantic Frame Parsing Model for Intent Detection and Slot Filling](https://mlsanity.com/paper/1812.10235/) (2018)
- [Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification](https://mlsanity.com/paper/2109.05427/) (2021)
- [A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives](https://mlsanity.com/paper/2102.12982/) (2021)
- [Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering](https://mlsanity.com/paper/2109.03381/) (2021)
- [GL-GIN: Fast and Accurate Non-Autoregressive Model for Joint Multiple Intent Detection and Slot Filling](https://mlsanity.com/paper/2106.01925/) (2021)
- [Pairwise Supervised Contrastive Learning of Sentence Representations](https://mlsanity.com/paper/2109.05424/) (2021)
- [Intent Detection and Slot Filling for Vietnamese](https://mlsanity.com/paper/2104.02021/) (2021)
- [Joint Multiple Intent Detection and Slot Filling via Self-distillation](https://mlsanity.com/paper/2108.08042/) (2021)
- [SLIM: Explicit Slot-Intent Mapping with BERT for Joint Multi-Intent Detection and Slot Filling](https://mlsanity.com/paper/2108.11711/) (2021)
- [A Scope Sensitive and Result Attentive Model for Multi-Intent Spoken Language Understanding](https://mlsanity.com/paper/2211.12220/) (2022)

## Built on by (most cited first, 7 of 7)

- [Audio2Tool: Speak, Call, Act -- A Dataset for Benchmarking Speech Tool Use](https://mlsanity.com/paper/2604.22821/) (2026)
- [Advancing Single and Multi-task Text Classification through Large Language Model Fine-tuning](https://mlsanity.com/paper/2412.08587/) (2024)
- [BlendX: Complex Multi-Intent Detection with Blended Patterns](https://mlsanity.com/paper/2403.18277/) (2024)
- [A Two-Stage Prediction-Aware Contrastive Learning Framework for Multi-Intent NLU](https://mlsanity.com/paper/2405.02925/) (2024)
- [MIDAS: Multi-level Intent, Domain, And Slot Knowledge Distillation for Multi-turn NLU](https://mlsanity.com/paper/2408.08144/) (2024)
- [Instance Relation Learning Network with Label Knowledge Propagation for Few-shot Multi-label Intent Detection](https://mlsanity.com/paper/2510.07776/) (2025)
- [Multi-Intent Spoken Language Understanding: Methods, Trends, and Challenges](https://mlsanity.com/paper/2512.11258/) (2025)

## Abstract (arXiv)

Multiple intent detection and slot filling are two fundamental and crucial tasks in spoken language understanding. Motivated by the fact that the two tasks are closely related, joint models that can detect intents and extract slots simultaneously are preferred to individual models that perform each task independently. The accuracy of a joint model depends heavily on the ability of the model to transfer information between the two tasks so that the result of one task can correct the result of the other. In addition, since a joint model has multiple outputs, how to train the model effectively is also challenging. In this paper, we present a method for multiple intent detection and slot filling by addressing these challenges. First, we propose a bidirectional joint model that explicitly employs intent information to recognize slots and slot features to detect intents. Second, we introduce a novel method for training the proposed joint model using supervised contrastive learning and self-distillation. Experimental results on two benchmark datasets MixATIS and MixSNIPS show that our method outperforms state-of-the-art models in both tasks. The results also demonstrate the contributions of both bidirectional design and the training method to the accuracy improvement. Our source code is available at https://github.com/anhtunguyen98/BiSLU