# A Co-Interactive Transformer for Joint Slot Filling and Intent Detection

arXiv: https://arxiv.org/abs/2010.03880 · HTML: https://mlsanity.com/paper/2010.03880/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2020.

## Built on by (most cited first, 30 of 61)

- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [A Survey on Spoken Language Understanding: Recent Advances and New Frontiers](https://mlsanity.com/paper/2103.03095/) (2021)
- [A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods](https://mlsanity.com/paper/2204.03508/) (2022)
- [GL-GIN: Fast and Accurate Non-Autoregressive Model for Joint Multiple Intent Detection and Slot Filling](https://mlsanity.com/paper/2106.01925/) (2021)
- [A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding](https://mlsanity.com/paper/2304.04256/) (2023)
- [Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System](https://mlsanity.com/paper/2109.11292/) (2021)
- [End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions](https://mlsanity.com/paper/2311.09008/) (2023)
- [MNER-QG: An End-to-End MRC framework for Multimodal Named Entity Recognition with Query Grounding](https://mlsanity.com/paper/2211.14739/) (2022)
- [GL-CLeF: A Global-Local Contrastive Learning Framework for Cross-lingual Spoken Language Understanding](https://mlsanity.com/paper/2204.08325/) (2022)
- [Co-guiding Net: Achieving Mutual Guidances between Multiple Intent Detection and Slot Filling via Heterogeneous Semantics-Label Graphs](https://mlsanity.com/paper/2210.10375/) (2022)
- [Towards Joint Intent Detection and Slot Filling via Higher-order Attention](https://mlsanity.com/paper/2109.08890/) (2021)
- [Group is better than individual: Exploiting Label Topologies and Label Relations for Joint Multiple Intent Detection and Slot Filling](https://mlsanity.com/paper/2210.10369/) (2022)
- [An Explicit-Joint and Supervised-Contrastive Learning Framework for Few-Shot Intent Classification and Slot Filling](https://mlsanity.com/paper/2110.13691/) (2021)
- [UniDU: Towards A Unified Generative Dialogue Understanding Framework](https://mlsanity.com/paper/2204.04637/) (2022)
- [A Dynamic Graph Interactive Framework with Label-Semantic Injection for Spoken Language Understanding](https://mlsanity.com/paper/2211.04023/) (2022)
- [Text is no more Enough! A Benchmark for Profile-based Spoken Language Understanding](https://mlsanity.com/paper/2112.11953/) (2021)
- [Relational Temporal Graph Reasoning for Dual-task Dialogue Language Understanding](https://mlsanity.com/paper/2306.09114/) (2023)
- [Joint Multiple Intent Detection and Slot Filling with Supervised Contrastive Learning and Self-Distillation](https://mlsanity.com/paper/2308.14654/) (2023)
- [S3-DST: Structured Open-Domain Dialogue Segmentation and State Tracking in the Era of LLMs](https://mlsanity.com/paper/2309.08827/) (2023)
- [ECLM: Entity Level Language Model for Spoken Language Understanding with Chain of Intent](https://mlsanity.com/paper/2403.04481/) (2024)
- [A Simple Meta-learning Paradigm for Zero-shot Intent Classification with Mixture Attention Mechanism](https://mlsanity.com/paper/2206.02179/) (2022)
- [An Effective Non-Autoregressive Model for Spoken Language Understanding](https://mlsanity.com/paper/2108.07005/) (2021)
- [Learning to Classify Open Intent via Soft Labeling and Manifold Mixup](https://mlsanity.com/paper/2204.07804/) (2022)
- [Explainable Slot Type Attentions to Improve Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/2210.10227/) (2022)
- [A Result based Portable Framework for Spoken Language Understanding](https://mlsanity.com/paper/2103.06010/) (2021)
- [Label-aware Multi-level Contrastive Learning for Cross-lingual Spoken Language Understanding](https://mlsanity.com/paper/2205.03656/) (2022)
- [A Fast Attention Network for Joint Intent Detection and Slot Filling on Edge Devices](https://mlsanity.com/paper/2205.07646/) (2022)
- [Intent Detection and Slot Filling for Home Assistants: Dataset and Analysis for Bangla and Sylheti](https://mlsanity.com/paper/2310.10935/) (2023)
- [CroPrompt: Cross-task Interactive Prompting for Zero-shot Spoken Language Understanding](https://mlsanity.com/paper/2406.10505/) (2024)
- [Capture Salient Historical Information: A Fast and Accurate Non-Autoregressive Model for Multi-turn Spoken Language Understanding](https://mlsanity.com/paper/2206.12209/) (2022)

## Abstract (arXiv)

Intent detection and slot filling are two main tasks for building a spoken language understanding (SLU) system. The two tasks are closely related and the information of one task can be utilized in the other task. Previous studies either model the two tasks separately or only consider the single information flow from intent to slot. None of the prior approaches model the bidirectional connection between the two tasks simultaneously. In this paper, we propose a Co-Interactive Transformer to consider the cross-impact between the two tasks. Instead of adopting the self-attention mechanism in vanilla Transformer, we propose a co-interactive module to consider the cross-impact by building a bidirectional connection between the two related tasks. In addition, the proposed co-interactive module can be stacked to incrementally enhance each other with mutual features. The experimental results on two public datasets (SNIPS and ATIS) show that our model achieves the state-of-the-art performance with considerable improvements (+3.4% and +0.9% on overall acc). Extensive experiments empirically verify that our model successfully captures the mutual interaction knowledge.