# Text Understanding from Scratch

arXiv: https://arxiv.org/abs/1502.01710 · HTML: https://mlsanity.com/paper/1502.01710/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Feb 2015.

## Built on by (most cited first, 25 of 25)

- [A Survey on Data Augmentation for Text Classification](https://mlsanity.com/paper/2107.03158/) (2021)
- [Supporting Clustering with Contrastive Learning](https://mlsanity.com/paper/2103.12953/) (2021)
- [On the Relation between Sensitivity and Accuracy in In-context Learning](https://mlsanity.com/paper/2209.07661/) (2022)
- [Pairwise Supervised Contrastive Learning of Sentence Representations](https://mlsanity.com/paper/2109.05424/) (2021)
- [Mitigating Memorization of Noisy Labels via Regularization between Representations](https://mlsanity.com/paper/2110.09022/) (2021)
- [Virtual Augmentation Supported Contrastive Learning of Sentence Representations](https://mlsanity.com/paper/2110.08552/) (2021)
- [NewsEmbed: Modeling News through Pre-trained Document Representations](https://mlsanity.com/paper/2106.00590/) (2021)
- [Contrastive Learning with Prompt-derived Virtual Semantic Prototypes for Unsupervised Sentence Embedding](https://mlsanity.com/paper/2211.03348/) (2022)
- [Clustering-Aware Negative Sampling for Unsupervised Sentence Representation](https://mlsanity.com/paper/2305.09892/) (2023)
- [Well-calibrated Confidence Measures for Multi-label Text Classification with a Large Number of Labels](https://mlsanity.com/paper/2312.09304/) (2023)
- [Explainable Artificial Intelligence (XAI) for Malware Analysis: A Survey of Techniques, Applications, and Open Challenges](https://mlsanity.com/paper/2409.13723/) (2024)
- [SocialInteractionGAN: Multi-person Interaction Sequence Generation](https://mlsanity.com/paper/2103.05916/) (2021)
- [Short Text Clustering with Transformers](https://mlsanity.com/paper/2102.00541/) (2021)
- [Large Language Models for Healthcare Text Classification: A Systematic Review](https://mlsanity.com/paper/2503.01159/) (2025)
- [How to Train Your Agent to Read and Write](https://mlsanity.com/paper/2101.00916/) (2021)
- [Adversarial Text Generation with Dynamic Contextual Perturbation](https://mlsanity.com/paper/2506.09148/) (2025)
- [The Role of the Input in Natural Language Video Description](https://mlsanity.com/paper/2102.05067/) (2021)
- [Contrastive Learning Subspace for Text Clustering](https://mlsanity.com/paper/2408.14119/) (2024)
- [Secondary Use of Clinical Problem List Entries for Neural Network-Based Disease Code Assignment](https://mlsanity.com/paper/2112.13756/) (2021)
- [CEIL: A General Classification-Enhanced Iterative Learning Framework for Text Clustering](https://mlsanity.com/paper/2304.11061/) (2023)
- [Mental Health Diagnosis in the Digital Age: Harnessing Sentiment Analysis on Social Media Platforms upon Ultra-Sparse Feature Content](https://mlsanity.com/paper/2311.05075/) (2023)
- [Understanding Student Sentiment on Mental Health Support in Colleges Using Large Language Models](https://mlsanity.com/paper/2412.04326/) (2024)
- [VizGen: Data Exploration and Visualization from Natural Language via a Multi-Agent AI Architecture](https://mlsanity.com/paper/2509.22218/) (2025)
- [PruneFuse: Efficient Data Selection via Weight Pruning and Network Fusion](https://mlsanity.com/paper/2603.26138/) (2026)
- [Random Label Prediction Heads for Studying Memorization in Deep Neural Networks](https://mlsanity.com/paper/2607.11541/) (2026)

## Abstract (arXiv)

This article demontrates that we can apply deep learning to text understanding from character-level inputs all the way up to abstract text concepts, using temporal convolutional networks (ConvNets). We apply ConvNets to various large-scale datasets, including ontology classification, sentiment analysis, and text categorization. We show that temporal ConvNets can achieve astonishing performance without the knowledge of words, phrases, sentences and any other syntactic or semantic structures with regards to a human language. Evidence shows that our models can work for both English and Chinese.