# Explainable Slot Type Attentions to Improve Joint Intent Detection and Slot Filling

arXiv: https://arxiv.org/abs/2210.10227 · HTML: https://mlsanity.com/paper/2210.10227/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 19 Oct 2022.

## Builds on (most cited first, 24 of 24)

- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019)
- [Explainable Recommendation: A Survey and New Perspectives](https://mlsanity.com/paper/1804.11192/) (2018)
- [A Survey of the State of Explainable AI for Natural Language Processing](https://mlsanity.com/paper/2010.00711/) (2020)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1609.01454/) (2016)
- [Self-Attention Attribution: Interpreting Information Interactions Inside Transformer](https://mlsanity.com/paper/2004.11207/) (2020)
- [A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding](https://mlsanity.com/paper/1909.02188/) (2019)
- [Joint Slot Filling and Intent Detection via Capsule Neural Networks](https://mlsanity.com/paper/1812.09471/) (2018)
- [A Co-Interactive Transformer for Joint Slot Filling and Intent Detection](https://mlsanity.com/paper/2010.03880/) (2020)
- [CM-Net: A Novel Collaborative Memory Network for Spoken Language Understanding](https://mlsanity.com/paper/1909.06937/) (2019)
- [Intent Detection and Slot Filling for Vietnamese](https://mlsanity.com/paper/2104.02021/) (2021)
- [Learning a Simple and Effective Model for Multi-turn Response Generation with Auxiliary Tasks](https://mlsanity.com/paper/2004.01972/) (2020)
- [Multi-Task Networks With Universe, Group, and Task Feature Learning](https://mlsanity.com/paper/1907.01791/) (2019)
- [Encoding Syntactic Knowledge in Transformer Encoder for Intent Detection and Slot Filling](https://mlsanity.com/paper/2012.11689/) (2020)
- [Using Neighborhood Context to Improve Information Extraction from Visual Documents Captured on Mobile Phones](https://mlsanity.com/paper/2108.10395/) (2021)
- [Coupled Representation Learning for Domains, Intents and Slots in Spoken Language Understanding](https://mlsanity.com/paper/1812.06083/) (2018)

## Built on by (most cited first, 5 of 5)

- [A Cross Attention Approach to Diagnostic Explainability using Clinical Practice Guidelines for Depression](https://mlsanity.com/paper/2311.13852/) (2023)
- [Towards Explainable and Safe Conversational Agents for Mental Health: A Survey](https://mlsanity.com/paper/2304.13191/) (2023)
- [MIDAS: Multi-level Intent, Domain, And Slot Knowledge Distillation for Multi-turn NLU](https://mlsanity.com/paper/2408.08144/) (2024)
- [Main Predicate and Their Arguments as Explanation Signals For Intent Classification](https://mlsanity.com/paper/2502.01270/) (2025)
- [Explainable and Accurate Natural Language Understanding for Voice Assistants and Beyond](https://mlsanity.com/paper/2309.14485/) (2023)

## Abstract (arXiv)

Joint intent detection and slot filling is a key research topic in natural language understanding (NLU). Existing joint intent and slot filling systems analyze and compute features collectively for all slot types, and importantly, have no way to explain the slot filling model decisions. In this work, we propose a novel approach that: (i) learns to generate additional slot type specific features in order to improve accuracy and (ii) provides explanations for slot filling decisions for the first time in a joint NLU model. We perform an additional constrained supervision using a set of binary classifiers for the slot type specific feature learning, thus ensuring appropriate attention weights are learned in the process to explain slot filling decisions for utterances. Our model is inherently explainable and does not need any post-hoc processing. We evaluate our approach on two widely used datasets and show accuracy improvements. Moreover, a detailed analysis is also provided for the exclusive slot explainability.