# Neural Attention Models for Sequence Classification: Analysis and Application to Key Term Extraction and Dialogue Act Detection

arXiv: https://arxiv.org/abs/1604.00077 · HTML: https://mlsanity.com/paper/1604.00077/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2016.

## Built on by (most cited first, 4 of 4)

- [SmartFRZ: An Efficient Training Framework using Attention-Based Layer Freezing](https://mlsanity.com/paper/2401.16720/) (2024)
- [A neural prosody encoder for end-ro-end dialogue act classification](https://mlsanity.com/paper/2205.05590/) (2022)
- [Classification of Alzheimers Disease with Deep Learning on Eye-tracking Data](https://mlsanity.com/paper/2309.12574/) (2023)
- [Channel-Imposed Fusion: A Simple yet Effective Method for Medical Time Series Classification](https://mlsanity.com/paper/2506.00337/) (2025)

## Abstract (arXiv)

Recurrent neural network architectures combining with attention mechanism, or neural attention model, have shown promising performance recently for the tasks including speech recognition, image caption generation, visual question answering and machine translation. In this paper, neural attention model is applied on two sequence classification tasks, dialogue act detection and key term extraction. In the sequence labeling tasks, the model input is a sequence, and the output is the label of the input sequence. The major difficulty of sequence labeling is that when the input sequence is long, it can include many noisy or irrelevant part. If the information in the whole sequence is treated equally, the noisy or irrelevant part may degrade the classification performance. The attention mechanism is helpful for sequence classification task because it is capable of highlighting important part among the entire sequence for the classification task. The experimental results show that with the attention mechanism, discernible improvements were achieved in the sequence labeling task considered here. The roles of the attention mechanism in the tasks are further analyzed and visualized in this paper.