mlsanityData to 14 Oct 2026

SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation

arXiv 1808.07512 ↗First posted 22 Aug 2018cs.CLcited by 72

Not on the map. Why?

Abstract (arXiv)

In this work, we examine methods for data augmentation for text-based tasks such as neural machine translation (NMT). We formulate the design of a data augmentation policy with desirable properties as an optimization problem, and derive a generic analytic solution. This solution not only subsumes some existing augmentation schemes, but also leads to an extremely simple data augmentation strategy for NMT: randomly replacing words in both the source sentence and the target sentence with other random words from their corresponding vocabularies. We name this method SwitchOut. Experiments on three translation datasets of different scales show that SwitchOut yields consistent improvements of about 0.5 BLEU, achieving better or comparable performances to strong alternatives such as word dropout (Sennrich et al., 2016a). Code to implement this method is included in the appendix.

Key facts. Cited by 72 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 24; 2022: 20; 2023: 17; 2024: 7; 2025: 2; 2026: 2.

72 papers.

Sort:

  1. 2021A Survey of Data Augmentation Approaches for NLPcited by 409
  2. 2021Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Bettercited by 154
  3. 2021A Survey on Data Augmentation for Text Classificationcited by 111
  4. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 109
  5. 2021Data Augmentation Approaches in Natural Language Processing: A Surveycited by 82
  6. 2021Advancing RNN Transducer Technology for Speech Recognitioncited by 44
  7. 2022Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Modelscited by 42
  8. 2021Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learningcited by 35
  9. 2021Substructure Substitution: Structured Data Augmentation for NLPcited by 25
  10. 2022TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understandingcited by 15
  11. 2021Syntactic Perturbations Reveal Representational Correlates of Hierarchical Phrase Structure in Pretrained Language Modelscited by 14
  12. 2022PCC: Paraphrasing with Bottom-k Sampling and Cyclic Learning for Curriculum Data Augmentationcited by 14
38 more
  1. 2023Towards energy-efficient Deep Learning: An overview of energy-efficient approaches along the Deep Learning Lifecyclecited by 14
  2. 2021Phrase-level Active Learning for Neural Machine Translationcited by 13
  3. 2022Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translationcited by 13
  4. 2021To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLPcited by 12
  5. 2021Data Augmentation by Concatenation for Low-Resource Translation: A Mystery and a Solutioncited by 11
  6. 2022Grammar-Based Grounded Lexicon Learningcited by 11
  7. 2023Model-Agnostic Meta-Learning for Multilingual Hate Speech Detectioncited by 11
  8. 2024Does DetectGPT Fully Utilize Perturbation? Bridging Selective Perturbation to Fine-tuned Contrastive Learning Detector would be Bettercited by 11
  9. 2021Data Augmentation for Low-Resource Named Entity Recognition Using Backtranslationcited by 10
  10. 2023GDA: Generative Data Augmentation Techniques for Relation Extraction Taskscited by 10
  11. 2021On-the-Fly Aligned Data Augmentation for Sequence-to-Sequence ASRcited by 9
  12. 2022CipherDAug: Ciphertext based Data Augmentation for Neural Machine Translationcited by 9
  13. 2022Exploring Temporally Dynamic Data Augmentation for Video Recognitioncited by 9
  14. 2021Text Augmentation in a Multi-Task Viewcited by 8
  15. 2021Sentence Concatenation Approach to Data Augmentation for Neural Machine Translationcited by 8
  16. 2021Rethinking Data Augmentation for Low-Resource Neural Machine Translation: A Multi-Task Learning Approachcited by 8
  17. 2022Compositionality as Lexical Symmetrycited by 8
  18. 2022BLISS: Robust Sequence-to-Sequence Learning via Self-Supervised Input Representationcited by 8
  19. 2024MEDSAGE: Enhancing Robustness of Medical Dialogue Summarization to ASR Errors with LLM-generated Synthetic Dialoguescited by 8
  20. 2025Overcoming Data Scarcity in Generative Language Modelling for Low-Resource Languages: A Systematic Reviewcited by 8
  21. 2023Symmetric Replay Training: Enhancing Sample Efficiency in Deep Reinforcement Learning for Combinatorial Optimizationcited by 7
  22. 2021Addressing the Vulnerability of NMT in Input Perturbationscited by 6
  23. 2024A Little Human Data Goes A Long Waycited by 6
  24. 2021Unsupervised Document Embedding via Contrastive Augmentationcited by 5
  25. 2023Target-Side Augmentation for Document-Level Machine Translationcited by 5
  26. 2023Neural Machine Translation Data Generation and Augmentation using ChatGPTcited by 5
  27. 2021AUGVIC: Exploiting BiText Vicinity for Low-Resource NMTcited by 4
  28. 2021Mitigating Data Scarceness through Data Synthesis, Augmentation and Curriculum for Abstractive Summarizationcited by 4
  29. 2022AdMix: A Mixed Sample Data Augmentation Method for Neural Machine Translationcited by 4
  30. 2022Scaling Back-Translation with Domain Text Generation for Sign Language Gloss Translationcited by 4
  31. 2023On Synthetic Data for Back Translationcited by 4
  32. 2024IndiText Boost: Text Augmentation for Low Resource India Languagescited by 4
  33. 2021Data Augmentation Methods for Anaphoric Zero Pronounscited by 3
  34. 2022Semantically Consistent Data Augmentation for Neural Machine Translation via Conditional Masked Language Modelcited by 3
  35. 2023Enhancing Robustness of AI Offensive Code Generators via Data Augmentationcited by 3
  36. 2021Reducing Exposure Bias in Training Recurrent Neural Network Transducerscited by 2
  37. 2022PAEG: Phrase-level Adversarial Example Generation for Neural Machine Translationcited by 2
  38. 2023Mitigating Data Scarcity for Large Language Modelscited by 2

50 most cited of 72 All 72 (JSON)