mlsanityData to 7 Oct 2026

SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness

arXiv 2009.10195 ↗First posted 21 Sep 2020cs.CL cs.LG stat.MLcited by 94

Not on the map. Why?

Abstract (arXiv)

Models that perform well on a training domain often fail to generalize to out-of-domain (OOD) examples. Data augmentation is a common method used to prevent overfitting and improve OOD generalization. However, in natural language, it is difficult to generate new examples that stay on the underlying data manifold. We introduce SSMBA, a data augmentation method for generating synthetic training examples by using a pair of corruption and reconstruction functions to move randomly on a data manifold. We investigate the use of SSMBA in the natural language domain, leveraging the manifold assumption to reconstruct corrupted text with masked language models. In experiments on robustness benchmarks across 3 tasks and 9 datasets, SSMBA consistently outperforms existing data augmentation methods and baseline models on both in-domain and OOD data, achieving gains of 0.8% accuracy on OOD Amazon reviews, 1.8% accuracy on OOD MNLI, and 1.4 BLEU on in-domain IWSLT14 German-English.

Key facts. Cited by 94 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 24; 2023: 27; 2024: 15; 2025: 7; 2026: 2.

94 papers.

Sort:

By area:

  1. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  2. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 185
  3. 2023Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventionscited by 121
  4. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  5. 2023Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluationscited by 96
  6. 2021Data Augmentation Approaches in Natural Language Processing: A Surveycited by 82
  7. 2021Towards Improving Adversarial Training of NLP Modelscited by 79
  8. 2021Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translationcited by 56
  9. 2022Perturbation Augmentation for Fairer NLPcited by 51
  10. 2021An Investigation of the (In)effectiveness of Counterfactually Augmented Datacited by 42
  11. 2021SSMix: Saliency-Based Span Mixup for Text Classificationcited by 39
  12. 2022DoCoGen: Domain Counterfactual Generation for Low Resource Domain Adaptationcited by 36
38 more
  1. 2021FlipDA: Effective and Robust Data Augmentation for Few-Shot Learningcited by 32
  2. 2022Improved Multi-label Classification under Temporal Concept Drift: Rethinking Group-Robust Algorithms in a Label-Wise Settingcited by 23
  3. 2023Mixture of Soft Prompts for Controllable Data Generationcited by 23
  4. 2022CoNT: Contrastive Neural Text Generationcited by 21
  5. 2021GOLD: Improving Out-of-Scope Detection in Dialogues using Data Augmentationcited by 20
  6. 2021Self-training Improves Pre-training for Few-shot Learning in Task-oriented Dialog Systemscited by 18
  7. 2024HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Modelscited by 17
  8. 2022TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understandingcited by 15
  9. 2022Improving the Sample Efficiency of Prompt Tuning with Domain Adaptationcited by 15
  10. 2021Learning to Perturb Word Embeddings for Out-of-distribution QAcited by 13
  11. 2022Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translationcited by 13
  12. 2024What Variables Affect Out-of-Distribution Generalization in Pretrained Models?cited by 13
  13. 2023ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Modelscited by 12
  14. 2022Estimating Soft Labels for Out-of-Domain Intent Detectioncited by 11
  15. 2021Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMaxcited by 10
  16. 2021ALP: Data Augmentation using Lexicalized PCFGs for Few-Shot Text Classificationcited by 10
  17. 2021On-the-Fly Aligned Data Augmentation for Sequence-to-Sequence ASRcited by 9
  18. 2021Consistency Training with Virtual Adversarial Discrete Perturbationcited by 9
  19. 2021Exploiting Position Bias for Robust Aspect Sentiment Classificationcited by 9
  20. 2022Impact of Adversarial Training on Robustness and Generalizability of Language Modelscited by 9
  21. 2024CoDa: Constrained Generation based Data Augmentation for Low-Resource NLPcited by 9
  22. 2022GeoECG: Data Augmentation via Wasserstein Geodesic Perturbation for Robust Electrocardiogram Predictioncited by 8
  23. 2024Improving Black-box Robustness with In-Context Rewritingcited by 8
  24. 2024ABEX: Data Augmentation for Low-Resource NLU via Expanding Abstract Descriptionscited by 8
  25. 2022EPiDA: An Easy Plug-in Data Augmentation Framework for High Performance Text Classificationcited by 7
  26. 2023Feature Programming for Multivariate Time Series Predictioncited by 7
  27. 2024Connect Later: Improving Fine-tuning for Robustness with Targeted Augmentationscited by 7
  28. 2021LINDA: Unsupervised Learning to Interpolate in Natural Language Processingcited by 6
  29. 2022DSLOB: A Synthetic Limit Order Book Dataset for Benchmarking Forecasting Algorithms under Distributional Shiftcited by 6
  30. 2022When Chosen Wisely, More Data Is What You Need: A Universal Sample-Efficient Strategy For Data Augmentationcited by 5
  31. 2023Masked Language Model Based Textual Adversarial Example Detectioncited by 5
  32. 2023Neural Machine Translation Data Generation and Augmentation using ChatGPTcited by 5
  33. 2024An Effective Deployment of Diffusion LM for Data Augmentation in Low-Resource Sentiment Classificationcited by 5
  34. 2022Predicting Out-of-Domain Generalization with Neighborhood Invariancecited by 4
  35. 2023"A Tale of Two Movements": Identifying and Comparing Perspectives in #BlackLivesMatter and #BlueLivesMatter Movements-related Tweets using Weakly Supervised Graph-based Structured Predictioncited by 4
  36. 2024Controllable and Diverse Data Augmentation with Large Language Model for Low-Resource Open-Domain Dialogue Generationcited by 4
  37. 2024Targeted Augmentation for Low-Resource Event Extractioncited by 4
  38. 2025Exploring Gradient-Guided Masked Language Model to Detect Textual Adversarial Attackscited by 4

50 most cited of 94 All 94 (JSON)