# Hierarchical Attentional Hybrid Neural Networks for Document Classification

arXiv: https://arxiv.org/abs/1901.06610 · HTML: https://mlsanity.com/paper/1901.06610/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jan 2019.

## Built on by (most cited first, 10 of 10)

- [Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2311.12351/) (2023)
- [H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequences](https://mlsanity.com/paper/2107.11906/) (2021)
- [Graph Neural Networks for Text Classification: A Survey](https://mlsanity.com/paper/2304.11534/) (2023)
- [Email Spam Detection Using Hierarchical Attention Hybrid Deep Learning Method](https://mlsanity.com/paper/2204.07390/) (2022)
- [On-Device Document Classification using multimodal features](https://mlsanity.com/paper/2101.01880/) (2021)
- [Sentiment Analysis with Deep Learning Models: A Comparative Study on a Decade of Sinhala Language Facebook Data](https://mlsanity.com/paper/2201.03941/) (2022)
- [Attention-based Bidirectional LSTM for Deceptive Opinion Spam Classification](https://mlsanity.com/paper/2112.14789/) (2021)
- [CNN-Trans-Enc: A CNN-Enhanced Transformer-Encoder On Top Of Static BERT representations for Document Classification](https://mlsanity.com/paper/2209.06344/) (2022)
- [It's FLAN time! Summing feature-wise latent representations for interpretability](https://mlsanity.com/paper/2106.10086/) (2021)
- [Using Language Models on Low-end Hardware](https://mlsanity.com/paper/2305.02350/) (2023)

## Abstract (arXiv)

Document classification is a challenging task with important applications. The deep learning approaches to the problem have gained much attention recently. Despite the progress, the proposed models do not incorporate the knowledge of the document structure in the architecture efficiently and not take into account the contexting importance of words and sentences. In this paper, we propose a new approach based on a combination of convolutional neural networks, gated recurrent units, and attention mechanisms for document classification tasks. The main contribution of this work is the use of convolution layers to extract more meaningful, generalizable and abstract features by the hierarchical representation. The proposed method in this paper improves the results of the current attention-based approaches for document classification.