# Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings

arXiv: https://arxiv.org/abs/1602.02373 · HTML: https://mlsanity.com/paper/1602.02373/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Feb 2016.

## Built on by (most cited first, 11 of 11)

- [ChatGPT: Jack of all trades, master of none](https://mlsanity.com/paper/2302.10724/) (2023)
- [ChatGraph: Interpretable Text Classification by Converting ChatGPT Knowledge to Graphs](https://mlsanity.com/paper/2305.03513/) (2023)
- [SciReviewGen: A Large-scale Dataset for Automatic Literature Review Generation](https://mlsanity.com/paper/2305.15186/) (2023)
- [Are the Multilingual Models Better? Improving Czech Sentiment with Transformers](https://mlsanity.com/paper/2108.10640/) (2021)
- [Deep Learning-based Sentiment Classification: A Comparative Survey](https://mlsanity.com/paper/2312.17253/) (2023)
- [Sparse-softmax: A Simpler and Faster Alternative Softmax Transformation](https://mlsanity.com/paper/2112.12433/) (2021)
- [Linear Transformations for Cross-lingual Sentiment Analysis](https://mlsanity.com/paper/2209.07244/) (2022)
- [Learning Sequence Representations by Non-local Recurrent Neural Memory](https://mlsanity.com/paper/2207.09710/) (2022)
- [Decoding the Alphabet Soup of Degrees in the United States Postsecondary Education System Through Hybrid Method: Database and Text Mining](https://mlsanity.com/paper/2309.13050/) (2023)
- [How Does Overparameterization Affect Features?](https://mlsanity.com/paper/2407.00968/) (2024)
- [A Critical Review on the Effectiveness and Privacy Threats of Membership Inference Attacks](https://mlsanity.com/paper/2603.22987/) (2026)

## Abstract (arXiv)

One-hot CNN (convolutional neural network) has been shown to be effective for text categorization (Johnson & Zhang, 2015). We view it as a special case of a general framework which jointly trains a linear model with a non-linear feature generator consisting of `text region embedding + pooling'. Under this framework, we explore a more sophisticated region embedding method using Long Short-Term Memory (LSTM). LSTM can embed text regions of variable (and possibly large) sizes, whereas the region size needs to be fixed in a CNN. We seek effective and efficient use of LSTM for this purpose in the supervised and semi-supervised settings. The best results were obtained by combining region embeddings in the form of LSTM and convolution layers trained on unlabeled data. The results indicate that on this task, embeddings of text regions, which can convey complex concepts, are more useful than embeddings of single words in isolation. We report performances exceeding the previous best results on four benchmark datasets.