mlsanityData to 7 Oct 2026

Repairing Pronouns in Translation with BERT-Based Post-Editing

arXiv 2103.12838 ↗First posted 23 Mar 2021cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Pronouns are important determinants of a text's meaning but difficult to translate. This is because pronoun choice can depend on entities described in previous sentences, and in some languages pronouns may be dropped when the referent is inferrable from the context. These issues can lead Neural Machine Translation (NMT) systems to make critical errors on pronouns that impair intelligibility and even reinforce gender bias. We investigate the severity of this pronoun issue, showing that (1) in some domains, pronoun choice can account for more than half of a NMT systems' errors, and (2) pronouns have a disproportionately large impact on perceived translation quality. We then investigate a possible solution: fine-tuning BERT on a pronoun prediction task using chunks of source-side sentences, then using the resulting classifier to repair the translations of an existing NMT model. We offer an initial case study of this approach for the Japanese-English language pair, observing that a small number of translations are significantly improved according to human evaluators.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 49 references that are in the map. First posted on arXiv 23 Mar 2021.

Builds on

22 of 49 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  4. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  5. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  6. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  7. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  8. 2019Evaluating Gender Bias in Machine Translationcited by 247
  9. 2019Automatically Neutralizing Subjective Bias in Textcited by 105
  10. 2018The Best of Both Worlds: Combining Recent Advances in Neural Machine Translationcited by 101
  11. 2019Getting Gender Right in Neural Machine Translationcited by 98
  12. 2018Context-Aware Neural Machine Translation Learns Anaphora Resolutioncited by 97
10 more
  1. 2017Neural Machine Translation with Extended Contextcited by 95
  2. 2018Document-Level Neural Machine Translation with Hierarchical Attention Networkscited by 85
  3. 2016Personalized Machine Translation: Preserving Original Author Traitscited by 47
  4. 2019Lost in Translation: Loss and Decay of Linguistic Richness in Machine Translationcited by 45
  5. 2017A Challenge Set Approach to Evaluating Machine Translationcited by 37
  6. 2020Can Automatic Post-Editing Improve NMT?cited by 17
  7. 2017JESC: Japanese-English Subtitle Corpuscited by 16
  8. 2019One Model to Learn Both: Zero Pronoun Prediction and Translationcited by 14
  9. 2019Findings of the 2016 WMT Shared Task on Cross-lingual Pronoun Predictioncited by 5
  10. 2019Exploiting Sentential Context for Neural Machine Translationcited by 2

Built on by

Not cited yet.