# Encoder-decoder with Focus-mechanism for Sequence Labelling Based Spoken Language Understanding

arXiv: https://arxiv.org/abs/1608.02097 · HTML: https://mlsanity.com/paper/1608.02097/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Aug 2016.

## Built on by (most cited first, 9 of 9)

- [Multi-task Active Learning for Pre-trained Transformer-based Models](https://mlsanity.com/paper/2208.05379/) (2022)
- [Few-shot Learning for Slot Tagging with Attentive Relational Network](https://mlsanity.com/paper/2103.02333/) (2021)
- [Joint Intent Detection And Slot Filling Based on Continual Learning Model](https://mlsanity.com/paper/2102.10905/) (2021)
- [Speech Understanding on Tiny Devices with A Learning Cache](https://mlsanity.com/paper/2311.18188/) (2023)
- [A BiRGAT Model for Multi-intent Spoken Language Understanding with Hierarchical Semantic Frames](https://mlsanity.com/paper/2402.18258/) (2024)
- [A Joint and Domain-Adaptive Approach to Spoken Language Understanding](https://mlsanity.com/paper/2107.11768/) (2021)
- [Traffic Event Detection as a Slot Filling Problem](https://mlsanity.com/paper/2109.06035/) (2021)
- [A Unified Framework for Slot based Response Generation in a Multimodal Dialogue System](https://mlsanity.com/paper/2305.17433/) (2023)
- [A survey of joint intent detection and slot-filling models in natural language understanding](https://mlsanity.com/paper/2101.08091/) (2021)

## Abstract (arXiv)

This paper investigates the framework of encoder-decoder with attention for sequence labelling based spoken language understanding. We introduce Bidirectional Long Short Term Memory - Long Short Term Memory networks (BLSTM-LSTM) as the encoder-decoder model to fully utilize the power of deep learning. In the sequence labelling task, the input and output sequences are aligned word by word, while the attention mechanism cannot provide the exact alignment. To address this limitation, we propose a novel focus mechanism for encoder-decoder framework. Experiments on the standard ATIS dataset showed that BLSTM-LSTM with focus mechanism defined the new state-of-the-art by outperforming standard BLSTM and attention based encoder-decoder. Further experiments also show that the proposed model is more robust to speech recognition errors.