# Low Resource Text Classification with ULMFit and Backtranslation

arXiv: https://arxiv.org/abs/1903.09244 · HTML: https://mlsanity.com/paper/1903.09244/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Mar 2019.

## Built on by (most cited first, 20 of 20)

- [An Empirical Survey of Data Augmentation for Limited Data Learning in NLP](https://mlsanity.com/paper/2106.07499/) (2021)
- [ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embedding](https://mlsanity.com/paper/2109.04380/) (2021)
- [Generate, Annotate, and Learn: NLP with Synthetic Text](https://mlsanity.com/paper/2106.06168/) (2021)
- [Text Smoothing: Enhance Various Data Augmentation Methods on Text Classification Tasks](https://mlsanity.com/paper/2202.13840/) (2022)
- [TOD-DA: Towards Boosting the Robustness of Task-oriented Dialogue Modeling on Spoken Conversations](https://mlsanity.com/paper/2112.12441/) (2021)
- [Data Augmentation in a Hybrid Approach for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2103.15912/) (2021)
- [Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks](https://mlsanity.com/paper/2305.13547/) (2023)
- [Bridging the Gap between Decision and Logits in Decision-based Knowledge Distillation for Pre-trained Language Models](https://mlsanity.com/paper/2306.08909/) (2023)
- [An Effective Deployment of Diffusion LM for Data Augmentation in Low-Resource Sentiment Classification](https://mlsanity.com/paper/2409.03203/) (2024)
- [STA: Self-controlled Text Augmentation for Improving Text Classifications](https://mlsanity.com/paper/2302.12784/) (2023)
- [Concept Extraction for Time Series with ECLAD-ts](https://mlsanity.com/paper/2504.05024/) (2025)
- [A Context-aware Attention and Graph Neural Network-based Multimodal Framework for Misogyny Detection](https://mlsanity.com/paper/2508.09175/) (2025)
- [Adversarial Word Dilution as Text Data Augmentation in Low-Resource Regime](https://mlsanity.com/paper/2305.09287/) (2023)
- [A Diversity-Enhanced and Constraints-Relaxed Augmentation for Low-Resource Classification](https://mlsanity.com/paper/2109.11834/) (2021)
- [Exploiting All Samples in Low-Resource Sentence Classification: Early Stopping and Initialization Parameters](https://mlsanity.com/paper/2111.06971/) (2021)
- [Improving Persian Relation Extraction Models by Data Augmentation](https://mlsanity.com/paper/2203.15323/) (2022)
- [Coping with low data availability for social media crisis message categorisation](https://mlsanity.com/paper/2305.17211/) (2023)
- [Investigating Masking-based Data Generation in Language Models](https://mlsanity.com/paper/2307.00008/) (2023)
- [Summarization-based Data Augmentation for Document Classification](https://mlsanity.com/paper/2312.00513/) (2023)
- [Advancing NLP Models with Strategic Text Augmentation: A Comprehensive Study of Augmentation Methods and Curriculum Strategies](https://mlsanity.com/paper/2402.09141/) (2024)

## Abstract (arXiv)

In computer vision, virtually every state-of-the-art deep learning system is trained with data augmentation. In text classification, however, data augmentation is less widely practiced because it must be performed before training and risks introducing label noise. We augment the IMDB movie reviews dataset with examples generated by two families of techniques: random token perturbations introduced by Wei and Zou [2019] and backtranslation -- translating to a second language then back to English. In low resource environments, backtranslation generates significant improvement on top of the state of-the-art ULMFit model. A ULMFit model pretrained on wikitext103 and then fine-tuned on only 50 IMDB examples and 500 synthetic examples generated by backtranslation achieves 80.6% accuracy, an 8.1% improvement over the augmentation-free baseline with only 9 minutes of additional training time. Random token perturbations do not yield any improvements but incur equivalent computational cost. The benefits of training with backtranslated examples decreases with the size of the available training data. On the full dataset, neither augmentation technique improves upon ULMFit's state of the art performance. We address this by using backtranslations as a form of test time augmentation as well as ensembling ULMFit with other models, and achieve small improvements.