# An Effective Deployment of Diffusion LM for Data Augmentation in Low-Resource Sentiment Classification

arXiv: https://arxiv.org/abs/2409.03203 · HTML: https://mlsanity.com/paper/2409.03203/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 5 Sep 2024.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [ERNIE 2.0: A Continual Pre-training Framework for Language Understanding](https://mlsanity.com/paper/1907.12412/) (2019)
- [DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models](https://mlsanity.com/paper/2211.15029/) (2022)
- [Not Enough Data? Deep Learning to the Rescue!](https://mlsanity.com/paper/1911.03118/) (2019)
- [Conditional BERT Contextual Augmentation](https://mlsanity.com/paper/1812.06705/) (2018)
- [SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness](https://mlsanity.com/paper/2009.10195/) (2020)
- [AEDA: An Easier Data Augmentation Technique for Text Classification](https://mlsanity.com/paper/2108.13230/) (2021)
- [Augmenting Low-Resource Text Classification with Graph-Grounded Pre-training and Prompting](https://mlsanity.com/paper/2305.03324/) (2023)
- [Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification](https://mlsanity.com/paper/1606.01614/) (2016)
- [Out-of-Domain Detection for Low-Resource Text Classification Tasks](https://mlsanity.com/paper/1909.05357/) (2019)
- [Low Resource Text Classification with ULMFit and Backtranslation](https://mlsanity.com/paper/1903.09244/) (2019)
- [SenWave: Monitoring the Global Sentiments under the COVID-19 Pandemic](https://mlsanity.com/paper/2006.10842/) (2020)
- [GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentation](https://mlsanity.com/paper/2211.10330/) (2022)
- [ALP: Data Augmentation using Lexicalized PCFGs for Few-Shot Text Classification](https://mlsanity.com/paper/2112.11916/) (2021)
- [Enhancing Low-Resource LLMs Classification with PEFT and Synthetic Data](https://mlsanity.com/paper/2404.02422/) (2024)
- [Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks](https://mlsanity.com/paper/2305.13547/) (2023)
- [Domain Adaptation for Sparse-Data Settings: What Do We Gain by Not Using Bert?](https://mlsanity.com/paper/2203.16926/) (2022)
- [Not Enough Labeled Data? Just Add Semantics: A Data-Efficient Method for Inferring Online Health Texts](https://mlsanity.com/paper/2309.09877/) (2023)

## Built on by (most cited first, 5 of 5)

- [Discrete Diffusion in Large Language and Multimodal Models: A Survey](https://mlsanity.com/paper/2506.13759/) (2025)
- [Debunk and Infer: Multimodal Fake News Detection via Diffusion-Generated Evidence and LLM Reasoning](https://mlsanity.com/paper/2506.21557/) (2025)
- [ProtoConNet: Prototypical Augmentation and Alignment for Open-Set Few-Shot Image Classification](https://mlsanity.com/paper/2507.11845/) (2025)
- [SimDiffRec: Semantic Similarity-Guided Diffusion for Contrastive Sequential Recommendation](https://mlsanity.com/paper/2507.11866/) (2025)
- [CoCoGEC: Counterfactual Generation for Robust Grammatical Error Correction](https://mlsanity.com/paper/2606.15069/) (2026)

## Abstract (arXiv)

Sentiment classification (SC) often suffers from low-resource challenges such as domain-specific contexts, imbalanced label distributions, and few-shot scenarios. The potential of the diffusion language model (LM) for textual data augmentation (DA) remains unexplored, moreover, textual DA methods struggle to balance the diversity and consistency of new samples. Most DA methods either perform logical modifications or rephrase less important tokens in the original sequence with the language model. In the context of SC, strong emotional tokens could act critically on the sentiment of the whole sequence. Therefore, contrary to rephrasing less important context, we propose DiffusionCLS to leverage a diffusion LM to capture in-domain knowledge and generate pseudo samples by reconstructing strong label-related tokens. This approach ensures a balance between consistency and diversity, avoiding the introduction of noise and augmenting crucial features of datasets. DiffusionCLS also comprises a Noise-Resistant Training objective to help the model generalize. Experiments demonstrate the effectiveness of our method in various low-resource scenarios including domain-specific and domain-general problems. Ablation studies confirm the effectiveness of our framework's modules, and visualization studies highlight optimal deployment conditions, reinforcing our conclusions.