mlsanityData to 7 Oct 2026

A Fast Attention Network for Joint Intent Detection and Slot Filling on Edge Devices

arXiv 2205.07646 ↗First posted 16 May 2022cs.CL cs.SD eess.AScited by 3

Not on the map. Why?

Abstract (arXiv)

Intent detection and slot filling are two main tasks in natural language understanding and play an essential role in task-oriented dialogue systems. The joint learning of both tasks can improve inference accuracy and is popular in recent works. However, most joint models ignore the inference latency and cannot meet the need to deploy dialogue systems at the edge. In this paper, we propose a Fast Attention Network (FAN) for joint intent detection and slot filling tasks, guaranteeing both accuracy and latency. Specifically, we introduce a clean and parameter-refined attention module to enhance the information exchange between intent and slot, improving semantic accuracy by more than 2%. FAN can be implemented on different encoders and delivers more accurate models at every speed level. Our experiments on the Jetson Nano platform show that FAN inferences fifteen utterances per second with a small accuracy drop, showing its effectiveness and efficiency on edge devices.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 32 of its 52 references that are in the map. First posted on arXiv 16 May 2022.

Builds on

32 of 52 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2016Layer Normalizationcited by 5,579
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  8. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  9. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  10. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  11. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  12. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
20 more
  1. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  2. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  3. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 476
  4. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 376
  5. 2021I-BERT: Integer-only BERT Quantizationcited by 250
  6. 2020FastBERT: a Self-distilling BERT with Adaptive Inference Timecited by 224
  7. 2019Structured Pruning of Large Language Modelscited by 201
  8. 2019BERT for Joint Intent Classification and Slot Fillingcited by 195
  9. 2016Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Fillingcited by 137
  10. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
  11. 2019A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understandingcited by 106
  12. 2019A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Fillingcited by 85
  13. 2020A Co-Interactive Transformer for Joint Slot Filling and Intent Detectioncited by 61
  14. 2018Explanation Methods in Deep Learning: Users, Values, Concerns and Challengescited by 49
  15. 2020SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Fillingcited by 42
  16. 2019Hierarchically-Refined Label Attention Network for Sequence Labelingcited by 25
  17. 2020Bringing AI To Edge: From Deep Learning's Perspectivecited by 24
  18. 2021A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networkscited by 14
  19. 2020Artificial Intelligence at the Edgecited by 2
  20. 2021Joint Intent Detection and Slot Filling with Wheel-Graph Attention Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 1.

3 papers.

  1. 2024MIDAS: Multi-level Intent, Domain, And Slot Knowledge Distillation for Multi-turn NLUcited by 2
  2. 2024Transforming NLU with Babylon: A Case Study in Development of Real-time, Edge-Efficient, Multi-Intent Translation System for Automated Drive-Thru Orderingcited by 1
  3. 2026ReacTOD: Bounded Neuro-Symbolic Agentic NLU for Zero-Shot Dialogue State Trackingcited by 1