# Sentence Concatenation Approach to Data Augmentation for Neural Machine Translation

arXiv: https://arxiv.org/abs/2104.08478 · HTML: https://mlsanity.com/paper/2104.08478/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 15 references that are in the map.
- First posted on arXiv 17 Apr 2021.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Six Challenges for Neural Machine Translation](https://mlsanity.com/paper/1706.03872/) (2017)
- [MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://mlsanity.com/paper/2004.12239/) (2020)
- [SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation](https://mlsanity.com/paper/1808.07512/) (2018)
- [Neural Machine Translation with Reconstruction](https://mlsanity.com/paper/1611.01874/) (2016)
- [Enabling Multi-Source Neural Machine Translation By Concatenating Source Sentences In Multiple Languages](https://mlsanity.com/paper/1702.06135/) (2017)

## Built on by (most cited first, 8 of 8)

- [Sequence Length is a Domain: Length-based Overfitting in Transformer Models](https://mlsanity.com/paper/2109.07276/) (2021)
- [Data Augmentation by Concatenation for Low-Resource Translation: A Mystery and a Solution](https://mlsanity.com/paper/2105.01691/) (2021)
- [SegAugment: Maximizing the Utility of Speech Translation Data with Segmentation-based Augmentations](https://mlsanity.com/paper/2212.09699/) (2022)
- [Checks and Strategies for Enabling Code-Switched Machine Translation](https://mlsanity.com/paper/2210.05096/) (2022)
- [Joint Speech Transcription and Translation: Pseudo-Labeling with Out-of-Distribution Data](https://mlsanity.com/paper/2212.09982/) (2022)
- [DSS: Synthesizing long Digital Ink using Data augmentation, Style encoding and Split generation](https://mlsanity.com/paper/2311.17786/) (2023)
- [Non-Fluent Synthetic Target-Language Data Improve Neural Machine Translation](https://mlsanity.com/paper/2401.16086/) (2024)
- [From Scarcity to Efficiency: Investigating the Effects of Data Augmentation on African Machine Translation](https://mlsanity.com/paper/2509.07471/) (2025)

## Abstract (arXiv)

Neural machine translation (NMT) has recently gained widespread attention because of its high translation accuracy. However, it shows poor performance in the translation of long sentences, which is a major issue in low-resource languages. It is assumed that this issue is caused by insufficient number of long sentences in the training data. Therefore, this study proposes a simple data augmentation method to handle long sentences. In this method, we use only the given parallel corpora as the training data and generate long sentences by concatenating two sentences. Based on the experimental results, we confirm improvements in long sentence translation by the proposed data augmentation method, despite its simplicity. Moreover, the translation quality is further improved by the proposed method, when combined with back-translation.