mlsanityData to 14 Oct 2026

Data Augmentation by Concatenation for Low-Resource Translation: A Mystery and a Solution

arXiv 2105.01691 ↗First posted 4 May 2021cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate the driving factors behind concatenation, a simple but effective data augmentation method for low-resource neural machine translation. Our experiments suggest that discourse context is unlikely the cause for the improvement of about +1 BLEU across four language pairs. Instead, we demonstrate that the improvement comes from three other factors unrelated to discourse: context diversity, length diversity, and (to a lesser extent) position shifting.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 18 of its 28 references that are in the map. First posted on arXiv 4 May 2021.

Builds on

18 of 28 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  3. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 306
  4. 2019Transformers without Tears: Improving the Normalization of Self-Attentioncited by 153
  5. 2017Data Augmentation for Low-Resource Neural Machine Translationcited by 139
  6. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 126
  7. 2017Neural Machine Translation with Extended Contextcited by 95
  8. 2018Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluationcited by 88
  9. 2018Improving the Transformer Translation Model with Document-Level Contextcited by 74
  10. 2018SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translationcited by 72
  11. 2019Microsoft Translator at WMT 2019: Towards Large-Scale Document-Level Neural Machine Translationcited by 56
  12. 2019Soft Contextual Data Augmentation for Neural Machine Translationcited by 49
6 more
  1. 2019When and Why is Document-level Context Useful in Neural Machine Translation?cited by 45
  2. 2017Does Neural Machine Translation Benefit from Larger Context?cited by 38
  3. 2018Neural Machine Translation of Text from Non-Native Speakerscited by 10
  4. 2021Sentence Concatenation Approach to Data Augmentation for Neural Machine Translationcited by 8
  5. 2020Using Context in Neural Machine Translation Training Objectivescited by 7
  6. 2017An Empirical Study of Mini-Batch Creation Strategies for Neural Machine Translationcited by 6

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 5; 2023: 3; 2024: 1.

11 papers.

  1. 2021NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentationcited by 73
  2. 2024Adapting Large Language Models for Document-Level Machine Translationcited by 59
  3. 2022A baseline revisited: Pushing the limits of multi-segment models for context-aware translationcited by 11
  4. 2021Characterizing and addressing the issue of oversmoothing in neural autoregressive sequence modelingcited by 10
  5. 2023Improving Long Context Document-Level Machine Translationcited by 8
  6. 2022Make More of Your Data: Minimal Effort Data Augmentation for Automatic Speech Recognition and Translationcited by 6
  7. 2022SegAugment: Maximizing the Utility of Speech Translation Data with Segmentation-based Augmentationscited by 6
  8. 2022Checks and Strategies for Enabling Code-Switched Machine Translationcited by 3
  9. 2022Joint Speech Transcription and Translation: Pseudo-Labeling with Out-of-Distribution Datacited by 3
  10. 2023End-to-End Single-Channel Speaker-Turn Aware Conversational Speech Translationcited by 2
  11. 2023On Search Strategies for Document-Level Neural Machine Translationcited by 0