# SSMix: Saliency-Based Span Mixup for Text Classification

arXiv: https://arxiv.org/abs/2106.08062 · HTML: https://mlsanity.com/paper/2106.08062/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 15 Jun 2021.

## Builds on (most cited first, 21 of 21)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks](https://mlsanity.com/paper/1905.11001/) (2019)
- [FreeLB: Enhanced Adversarial Training for Natural Language Understanding](https://mlsanity.com/paper/1909.11764/) (2019)
- [Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup](https://mlsanity.com/paper/2009.06962/) (2020)
- [Visualizing and Understanding Neural Models in NLP](https://mlsanity.com/paper/1506.01066/) (2015)
- [MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://mlsanity.com/paper/2004.12239/) (2020)
- [SaliencyMix: A Saliency Guided Data Augmentation Strategy for Better Regularization](https://mlsanity.com/paper/2006.01791/) (2020)
- [Augmenting Data with Mixup for Sentence Classification: An Empirical Study](https://mlsanity.com/paper/1905.08941/) (2019)
- [SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness](https://mlsanity.com/paper/2009.10195/) (2020)
- [NSML: Meet the MLaaS platform with a real-world case study](https://mlsanity.com/paper/1810.09957/) (2018)
- [NSML: A Machine Learning Platform That Enables You to Focus on Your Models](https://mlsanity.com/paper/1712.05902/) (2017)
- [Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Data](https://mlsanity.com/paper/1909.09148/) (2019)
- [Consistency Training with Virtual Adversarial Discrete Perturbation](https://mlsanity.com/paper/2104.07284/) (2021)

## Built on by (most cited first, 30 of 39)

- [Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Module](https://mlsanity.com/paper/2209.02604/) (2022)
- [A Comprehensive Survey on Data Augmentation](https://mlsanity.com/paper/2405.09591/) (2024)
- [On the Calibration of Pre-trained Language Models using Mixup Guided by Area Under the Margin and Saliency](https://mlsanity.com/paper/2203.07559/) (2022)
- [A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainability](https://mlsanity.com/paper/2212.10888/) (2022)
- [Geodesic Multi-Modal Mixup for Robust Fine-Tuning](https://mlsanity.com/paper/2203.03897/) (2022)
- [On the Domain Adaptation and Generalization of Pretrained Language Models: A Survey](https://mlsanity.com/paper/2211.03154/) (2022)
- [A Survey on Mixup Augmentations and Beyond](https://mlsanity.com/paper/2409.05202/) (2024)
- [Finding Order in Chaos: A Novel Data Augmentation Method for Time Series in Contrastive Learning](https://mlsanity.com/paper/2309.13439/) (2023)
- [TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understanding](https://mlsanity.com/paper/2205.06153/) (2022)
- [Improving the Sample Efficiency of Prompt Tuning with Domain Adaptation](https://mlsanity.com/paper/2210.02952/) (2022)
- [DoubleMix: Simple Interpolation-Based Data Augmentation for Text Classification](https://mlsanity.com/paper/2209.05297/) (2022)
- [SelfMix: Robust Learning Against Textual Label Noise with Self-Mixup Training](https://mlsanity.com/paper/2210.04525/) (2022)
- [Interdisciplinary Fairness in Imbalanced Research Proposal Topic Inference: A Hierarchical Transformer-based Method with Selective Interpolation](https://mlsanity.com/paper/2309.01717/) (2023)
- [LINDA: Unsupervised Learning to Interpolate in Natural Language Processing](https://mlsanity.com/paper/2112.13969/) (2021)
- [MixKG: Mixing for harder negative samples in knowledge graph](https://mlsanity.com/paper/2202.09606/) (2022)
- [Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks](https://mlsanity.com/paper/2305.13547/) (2023)
- [SimPSI: A Simple Strategy to Preserve Spectral Information in Time Series Data Augmentation](https://mlsanity.com/paper/2312.05790/) (2023)
- [MIXCODE: Enhancing Code Classification by Mixup-Based Data Augmentation](https://mlsanity.com/paper/2210.03003/) (2022)
- [Beyond Instance Discrimination: Relation-aware Contrastive Self-supervised Learning](https://mlsanity.com/paper/2211.01796/) (2022)
- [Data Augmentation for Conversational AI](https://mlsanity.com/paper/2309.04739/) (2023)
- [SMOTExT: SMOTE meets Large Language Models](https://mlsanity.com/paper/2505.13434/) (2025)
- [LM-mixup: Text Data Augmentation via Language Model based Mixup](https://mlsanity.com/paper/2510.20449/) (2025)
- [On the Effectiveness of Hybrid Pooling in Mixup-Based Graph Learning for Language Processing](https://mlsanity.com/paper/2210.03123/) (2022)
- [SMSMix: Sense-Maintained Sentence Mixup for Word Sense Disambiguation](https://mlsanity.com/paper/2212.07072/) (2022)
- [AttentionMix: Data augmentation method that relies on BERT attention mechanism](https://mlsanity.com/paper/2309.11104/) (2023)
- [ICON: Improving Inter-Report Consistency in Radiology Report Generation via Lesion-aware Mixup Augmentation](https://mlsanity.com/paper/2402.12844/) (2024)
- [Scan-and-Print: Patch-level Data Summarization and Augmentation for Content-aware Layout Generation in Poster Design](https://mlsanity.com/paper/2505.20649/) (2025)
- [Mitigating Shortcut Learning with InterpoLated Learning](https://mlsanity.com/paper/2507.05527/) (2025)
- [Hierarchical MixUp Multi-label Classification with Imbalanced Interdisciplinary Research Proposals](https://mlsanity.com/paper/2209.13912/) (2022)
- [AMPLIFY:Attention-based Mixup for Performance Improvement and Label Smoothing in Transformer](https://mlsanity.com/paper/2309.12689/) (2023)

## Abstract (arXiv)

Data augmentation with mixup has shown to be effective on various computer vision tasks. Despite its great success, there has been a hurdle to apply mixup to NLP tasks since text consists of discrete tokens with variable length. In this work, we propose SSMix, a novel mixup method where the operation is performed on input text rather than on hidden vectors like previous approaches. SSMix synthesizes a sentence while preserving the locality of two original texts by span-based mixing and keeping more tokens related to the prediction relying on saliency information. With extensive experiments, we empirically validate that our method outperforms hidden-level mixup methods on a wide range of text classification benchmarks, including textual entailment, sentiment classification, and question-type classification. Our code is available at https://github.com/clovaai/ssmix.