# Netmarble AI Center's WMT21 Automatic Post-Editing Shared Task Submission

arXiv: https://arxiv.org/abs/2109.06515 · HTML: https://mlsanity.com/paper/2109.06515/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 14 Sep 2021.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Unsupervised Quality Estimation for Neural Machine Translation](https://mlsanity.com/paper/2005.10608/) (2020)
- [Facebook FAIR's WMT19 News Translation Task Submission](https://mlsanity.com/paper/1907.06616/) (2019)
- [SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings](https://mlsanity.com/paper/2004.08728/) (2020)
- [Multi-task Learning for Multilingual Neural Machine Translation](https://mlsanity.com/paper/2010.02523/) (2020)
- [Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection](https://mlsanity.com/paper/2009.04703/) (2020)
- [eSCAPE: a Large-scale Synthetic Corpus for Automatic Post-Editing](https://mlsanity.com/paper/1803.07274/) (2018)
- [Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing](https://mlsanity.com/paper/1605.04800/) (2016)
- [Unbabel's Submission to the WMT2019 APE Shared Task: BERT-based Encoder-Decoder for Automatic Post-Editing](https://mlsanity.com/paper/1905.13068/) (2019)
- [Deep Context- and Relation-Aware Learning for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2106.03806/) (2021)

## Built on by (most cited first, 5 of 5)

- [Giving the Old a Fresh Spin: Quality Estimation-Assisted Constrained Decoding for Automatic Post-Editing](https://mlsanity.com/paper/2501.17265/) (2025)
- [PePe: Personalized Post-editing Model utilizing User-generated Post-edits](https://mlsanity.com/paper/2209.10139/) (2022)
- [Together We Can: Multilingual Automatic Post-Editing for Low-Resource Languages](https://mlsanity.com/paper/2410.17973/) (2024)
- [An Empirical Study of Automatic Post-Editing](https://mlsanity.com/paper/2209.07759/) (2022)
- [APE-then-QE: Correcting then Filtering Pseudo Parallel Corpora for MT Training Data Creation](https://mlsanity.com/paper/2312.11312/) (2023)

## Abstract (arXiv)

This paper describes Netmarble's submission to WMT21 Automatic Post-Editing (APE) Shared Task for the English-German language pair. First, we propose a Curriculum Training Strategy in training stages. Facebook Fair's WMT19 news translation model was chosen to engage the large and powerful pre-trained neural networks. Then, we post-train the translation model with different levels of data at each training stages. As the training stages go on, we make the system learn to solve multiple tasks by adding extra information at different training stages gradually. We also show a way to utilize the additional data in large volume for APE tasks. For further improvement, we apply Multi-Task Learning Strategy with the Dynamic Weight Average during the fine-tuning stage. To fine-tune the APE corpus with limited data, we add some related subtasks to learn a unified representation. Finally, for better performance, we leverage external translations as augmented machine translation (MT) during the post-training and fine-tuning. As experimental results show, our APE system significantly improves the translations of provided MT results by -2.848 and +3.74 on the development dataset in terms of TER and BLEU, respectively. It also demonstrates its effectiveness on the test dataset with higher quality than the development dataset.