# Effective Use of Word Order for Text Categorization with Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1412.1058 · HTML: https://mlsanity.com/paper/1412.1058/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2014.

## Built on by (most cited first, 30 of 40)

- [Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks](https://mlsanity.com/paper/2305.10160/) (2023)
- [Multi-Label Learning from Single Positive Labels](https://mlsanity.com/paper/2106.09708/) (2021)
- [Incorporating Hierarchy into Text Encoder: a Contrastive Learning Approach for Hierarchical Text Classification](https://mlsanity.com/paper/2203.03825/) (2022)
- [HPT: Hierarchy-aware Prompt Tuning for Hierarchical Text Classification](https://mlsanity.com/paper/2204.13413/) (2022)
- [HTCInfoMax: A Global Model for Hierarchical Text Classification via Information Maximization](https://mlsanity.com/paper/2104.05220/) (2021)
- [SciReviewGen: A Large-scale Dataset for Automatic Literature Review Generation](https://mlsanity.com/paper/2305.15186/) (2023)
- [A Transformer-based Framework for POI-level Social Post Geolocation](https://mlsanity.com/paper/2211.01336/) (2022)
- [Joint Learning of Hyperbolic Label Embeddings for Hierarchical Multi-label Classification](https://mlsanity.com/paper/2101.04997/) (2021)
- [Encoded Summarization: Summarizing Documents into Continuous Vector Space for Legal Case Retrieval](https://mlsanity.com/paper/2309.08187/) (2023)
- [TextConvoNet:A Convolutional Neural Network based Architecture for Text Classification](https://mlsanity.com/paper/2203.05173/) (2022)
- [Hierarchical Text Classification of Urdu News using Deep Neural Network](https://mlsanity.com/paper/2107.03141/) (2021)
- [Spam Review Detection Using Deep Learning](https://mlsanity.com/paper/2211.01675/) (2022)
- [Deep Learning-based Sentiment Classification: A Comparative Survey](https://mlsanity.com/paper/2312.17253/) (2023)
- [A Survey on Automated Sarcasm Detection on Twitter](https://mlsanity.com/paper/2202.02516/) (2022)
- [Comparing Baseline Shapley and Integrated Gradients for Local Explanation: Some Additional Insights](https://mlsanity.com/paper/2208.06096/) (2022)
- [Utilizing Local Hierarchy with Adversarial Training for Hierarchical Text Classification](https://mlsanity.com/paper/2402.18825/) (2024)
- [Literature review on vulnerability detection using NLP technology](https://mlsanity.com/paper/2104.11230/) (2021)
- [State-Regularized Recurrent Neural Networks to Extract Automata and Explain Predictions](https://mlsanity.com/paper/2212.05178/) (2022)
- [AE-smnsMLC: Multi-Label Classification with Semantic Matching and Negative Label Sampling for Product Attribute Value Extraction](https://mlsanity.com/paper/2310.07137/) (2023)
- [Preliminary Steps Towards Federated Sentiment Classification](https://mlsanity.com/paper/2107.11956/) (2021)
- [Hierarchy-Aware T5 with Path-Adaptive Mask Mechanism for Hierarchical Text Classification](https://mlsanity.com/paper/2109.08585/) (2021)
- [Solar cell patent classification method based on keyword extraction and deep neural network](https://mlsanity.com/paper/2109.08796/) (2021)
- [Open Problems in Applied Deep Learning](https://mlsanity.com/paper/2301.11316/) (2023)
- [JPAVE: A Generation and Classification-based Model for Joint Product Attribute Prediction and Value Extraction](https://mlsanity.com/paper/2311.04196/) (2023)
- [SECNN: Squeeze-and-Excitation Convolutional Neural Network for Sentence Classification](https://mlsanity.com/paper/2312.06088/) (2023)
- [Understanding Mental Health Content on Social Media and Its Effect Towards Suicidal Ideation](https://mlsanity.com/paper/2501.09309/) (2025)
- [Sent2Matrix: Folding Character Sequences in Serpentine Manifolds for Two-Dimensional Sentence](https://mlsanity.com/paper/2103.08387/) (2021)
- [Structural block driven - enhanced convolutional neural representation for relation extraction](https://mlsanity.com/paper/2103.11356/) (2021)
- [Automated Detection of GDPR Disclosure Requirements in Privacy Policies using Deep Active Learning](https://mlsanity.com/paper/2111.04224/) (2021)
- [TraceNet: Tracing and Locating the Key Elements in Sentiment Analysis](https://mlsanity.com/paper/2202.13812/) (2022)

## Abstract (arXiv)

Convolutional neural network (CNN) is a neural network that can make use of the internal structure of data such as the 2D structure of image data. This paper studies CNN on text categorization to exploit the 1D structure (namely, word order) of text data for accurate prediction. Instead of using low-dimensional word vectors as input as is often done, we directly apply CNN to high-dimensional text data, which leads to directly learning embedding of small text regions for use in classification. In addition to a straightforward adaptation of CNN from image to text, a simple but new variation which employs bag-of-word conversion in the convolution layer is proposed. An extension to combine multiple convolution layers is also explored for higher accuracy. The experiments demonstrate the effectiveness of our approach in comparison with state-of-the-art methods.