# Addressing Exposure Bias With Document Minimum Risk Training: Cambridge at the WMT20 Biomedical Translation Task

arXiv: https://arxiv.org/abs/2010.05333 · HTML: https://mlsanity.com/paper/2010.05333/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2020.

## Built on by (most cited first, 8 of 8)

- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Improving Automated Program Repair with Domain Adaptation](https://mlsanity.com/paper/2212.11414/) (2022)
- [Measuring Harmful Representations in Scandinavian Language Models](https://mlsanity.com/paper/2211.11678/) (2022)
- [Measuring Normative and Descriptive Biases in Language Models Using Census Data](https://mlsanity.com/paper/2304.05764/) (2023)
- [Domain adapted machine translation: What does catastrophic forgetting forget and why?](https://mlsanity.com/paper/2412.17537/) (2024)
- [Learning Horn Envelopes via Queries from Large Language Models](https://mlsanity.com/paper/2305.12143/) (2023)
- [Epi-Curriculum: Episodic Curriculum Learning for Low-Resource Domain Adaptation in Neural Machine Translation](https://mlsanity.com/paper/2309.02640/) (2023)
- [Enhancing Small Language Models for Power Outage Report Generation via Minimum Risk Training](https://mlsanity.com/paper/2609.27197/) (2026)

## Abstract (arXiv)

The 2020 WMT Biomedical translation task evaluated Medline abstract translations. This is a small-domain translation task, meaning limited relevant training data with very distinct style and vocabulary. Models trained on such data are susceptible to exposure bias effects, particularly when training sentence pairs are imperfect translations of each other. This can result in poor behaviour during inference if the model learns to neglect the source sentence. The UNICAM entry addresses this problem during fine-tuning using a robust variant on Minimum Risk Training. We contrast this approach with data-filtering to remove `problem' training examples. Under MRT fine-tuning we obtain good results for both directions of English-German and English-Spanish biomedical translation. In particular we achieve the best English-to-Spanish translation result and second-best Spanish-to-English result, despite using only single models with no ensembling.