# An Attention Ensemble Approach for Efficient Text Classification of Indian Languages

arXiv: https://arxiv.org/abs/2102.10275 · HTML: https://mlsanity.com/paper/2102.10275/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 31 references that are in the map.
- First posted on arXiv 20 Feb 2021.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Recurrent Neural Network for Text Classification with Multi-Task Learning](https://mlsanity.com/paper/1605.05101/) (2016)
- [Deep Learning for Hindi Text Classification: A Comparison](https://mlsanity.com/paper/2001.10340/) (2020)
- [Do Convolutional Networks need to be Deep for Text Classification ?](https://mlsanity.com/paper/1707.04108/) (2017)
- [Efficient Character-level Document Classification by Combining Convolution and Recurrent Layers](https://mlsanity.com/paper/1602.00367/) (2016)
- [Multichannel CNN with Attention for Text Classification](https://mlsanity.com/paper/2006.16174/) (2020)

## Built on by (most cited first, 1 of 1)

- [SECNN: Squeeze-and-Excitation Convolutional Neural Network for Sentence Classification](https://mlsanity.com/paper/2312.06088/) (2023)

## Abstract (arXiv)

The recent surge of complex attention-based deep learning architectures has led to extraordinary results in various downstream NLP tasks in the English language. However, such research for resource-constrained and morphologically rich Indian vernacular languages has been relatively limited. This paper proffers team SPPUAKAH's solution for the TechDOfication 2020 subtask-1f: which focuses on the coarse-grained technical domain identification of short text documents in Marathi, a Devanagari script-based Indian language. Availing the large dataset at hand, a hybrid CNN-BiLSTM attention ensemble model is proposed that competently combines the intermediate sentence representations generated by the convolutional neural network and the bidirectional long short-term memory, leading to efficient text classification. Experimental results show that the proposed model outperforms various baseline machine learning and deep learning models in the given task, giving the best validation accuracy of 89.57% and f1-score of 0.8875. Furthermore, the solution resulted in the best system submission for this subtask, giving a test accuracy of 64.26% and f1-score of 0.6157, transcending the performances of other teams as well as the baseline system given by the organizers of the shared task.