# Unsupervised Data Augmentation with Naive Augmentation and without Unlabeled Data

arXiv: https://arxiv.org/abs/2010.11966 · HTML: https://mlsanity.com/paper/2010.11966/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2020.

## Built on by (most cited first, 11 of 11)

- [Data Augmentation Approaches in Natural Language Processing: A Survey](https://mlsanity.com/paper/2110.01852/) (2021)
- [Prompt Perturbation Consistency Learning for Robust Language Models](https://mlsanity.com/paper/2402.15833/) (2024)
- [ConNER: Consistency Training for Cross-lingual Named Entity Recognition](https://mlsanity.com/paper/2211.09394/) (2022)
- [ConsistTL: Modeling Consistency in Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2212.04262/) (2022)
- [Joint Span Segmentation and Rhetorical Role Labeling with Data Augmentation for Legal Documents](https://mlsanity.com/paper/2302.06448/) (2023)
- [Mitigating Data Scarcity for Large Language Models](https://mlsanity.com/paper/2302.01806/) (2023)
- [Pseudo-labeling with Keyword Refining for Few-Supervised Video Captioning](https://mlsanity.com/paper/2411.04059/) (2024)
- [Vision-Enhanced Semantic Entity Recognition in Document Images via Visually-Asymmetric Consistency Learning](https://mlsanity.com/paper/2310.14785/) (2023)
- [Consistency Matters: Explore LLMs Consistency From a Black-Box Perspective](https://mlsanity.com/paper/2402.17411/) (2024)
- [Investigating Masking-based Data Generation in Language Models](https://mlsanity.com/paper/2307.00008/) (2023)
- [SCAT: Robust Self-supervised Contrastive Learning via Adversarial Training for Text Classification](https://mlsanity.com/paper/2307.01488/) (2023)

## Abstract (arXiv)

Unsupervised Data Augmentation (UDA) is a semi-supervised technique that applies a consistency loss to penalize differences between a model's predictions on (a) observed (unlabeled) examples; and (b) corresponding 'noised' examples produced via data augmentation. While UDA has gained popularity for text classification, open questions linger over which design decisions are necessary and over how to extend the method to sequence labeling tasks. This method has recently gained traction for text classification. In this paper, we re-examine UDA and demonstrate its efficacy on several sequential tasks. Our main contribution is an empirical study of UDA to establish which components of the algorithm confer benefits in NLP. Notably, although prior work has emphasized the use of clever augmentation techniques including back-translation, we find that enforcing consistency between predictions assigned to observed and randomly substituted words often yields comparable (or greater) benefits compared to these complex perturbation models. Furthermore, we find that applying its consistency loss affords meaningful gains without any unlabeled data at all, i.e., in a standard supervised setting. In short: UDA need not be unsupervised, and does not require complex data augmentation to be effective.