# TextCNN with Attention for Text Classification

arXiv: https://arxiv.org/abs/2108.01921 · HTML: https://mlsanity.com/paper/2108.01921/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 5 references that are in the map.
- First posted on arXiv 4 Aug 2021.

## Builds on (most cited first, 4 of 4)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Efficient Object Localization Using Convolutional Networks](https://mlsanity.com/paper/1411.4280/) (2014)

## Built on by (most cited first, 1 of 1)

- [SEMI-FND: Stacked Ensemble Based Multimodal Inference For Faster Fake News Detection](https://mlsanity.com/paper/2205.08159/) (2022)

## Abstract (arXiv)

The vast majority of textual content is unstructured, making automated classification an important task for many applications. The goal of text classification is to automatically classify text documents into one or more predefined categories. Recently proposed simple architectures for text classification such as Convolutional Neural Networks for Sentence Classification by Kim, Yoon showed promising results. In this paper, we propose incorporating an attention mechanism into the network to boost its performance, we also propose WordRank for vocabulary selection to reduce the network embedding parameters and speed up training with minimum accuracy loss. By adopting the proposed ideas TextCNN accuracy on 20News increased from 94.79 to 96.88, moreover, the number of parameters for the embedding layer can be reduced substantially with little accuracy loss by using WordRank. By using WordRank for vocabulary selection we can reduce the number of parameters by more than 5x from 7.9M to 1.5M, and the accuracy will only decrease by 1.2%.