mlsanityData to 7 Oct 2026

Deterministic Reversible Data Augmentation for Neural Machine Translation

arXiv 2406.02517 ↗First posted 4 Jun 2024cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Data augmentation is an effective way to diversify corpora in machine translation, but previous methods may introduce semantic inconsistency between original and augmented data because of irreversible operations and random subword sampling procedures. To generate both symbolically diverse and semantically consistent augmentation data, we propose Deterministic Reversible Data Augmentation (DRDA), a simple but effective data augmentation method for neural machine translation. DRDA adopts deterministic segmentations and reversible operations to generate multi-granularity subword representations and pulls them closer together with multi-view techniques. With no extra corpora or model changes required, DRDA outperforms strong baselines on several translation tasks with a clear margin (up to 4.3 BLEU gain over Transformer) and exhibits good robustness in noisy, low-resource, and cross-domain datasets.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 4 Jun 2024.

Builds on

32 of 43 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  6. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  7. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  8. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  9. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  10. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 304
  11. 2021R-Drop: Regularized Dropout for Neural Networkscited by 202
  12. 2019BPE-Dropout: Simple and Effective Subword Regularizationcited by 189
20 more
  1. 2020Byte Pair Encoding is Suboptimal for Language Model Pretrainingcited by 189
  2. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 125
  3. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 108
  4. 2019Robust Neural Machine Translation with Doubly Adversarial Inputscited by 105
  5. 2018Rapid Adaptation of Neural Machine Translation to New Languagescited by 82
  6. 2016Reward Augmented Maximum Likelihood for Neural Structured Predictioncited by 80
  7. 2016Edinburgh Neural Machine Translation Systems for WMT 16cited by 76
  8. 2018SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translationcited by 72
  9. 2020A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generationcited by 72
  10. 2020FLAT: Chinese NER Using Flat-Lattice Transformercited by 67
  11. 2020AdvAug: Robust Adversarial Augmentation for Neural Machine Translationcited by 66
  12. 2019Generalized Data Augmentation for Low-Resource Translationcited by 53
  13. 2019compare-mt: A Tool for Holistic Comparison of Language Generation Systemscited by 43
  14. 2021Multi-view Subword Regularizationcited by 40
  15. 2016Semi-Supervised Learning for Neural Machine Translationcited by 36
  16. 2019Multilingual Neural Machine Translation With Soft Decoupled Encodingcited by 21
  17. 2021Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Datacited by 17
  18. 2020Character-Level Translation with Self-attentioncited by 12
  19. 2019Multi-Granularity Self-Attention for Neural Machine Translationcited by 10
  20. 2014Semantic Composition and Decomposition: From Recognition to Generationcited by 3

Built on by

Not cited yet.