mlsanityData to 7 Oct 2026

Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation

arXiv 2105.15087 ↗First posted 31 May 2021cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

While it has been shown that Neural Machine Translation (NMT) is highly sensitive to noisy parallel training samples, prior work treats all types of mismatches between source and target as noise. As a result, it remains unclear how samples that are mostly equivalent but contain a small number of semantically divergent tokens impact NMT training. To close this gap, we analyze the impact of different types of fine-grained semantic divergences on Transformer models. We show that models trained on synthetic divergences output degenerated text more frequently and are less confident in their predictions. Based on these findings, we introduce a divergent-aware NMT framework that uses factors to help NMT recover from the degradation caused by naturally occurring divergences, improving both translation quality and model calibration on EN-FR tasks.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 23 of its 53 references that are in the map. First posted on arXiv 31 May 2021.

Builds on

23 of 53 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  7. 2019When Does Label Smoothing Help?cited by 985
  8. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  9. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  10. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 221
  11. 2018Analyzing Uncertainty in Neural Machine Translationcited by 126
  12. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 125
11 more
  1. 2018On the Impact of Various Types of Noise on Neural Machine Translationcited by 76
  2. 2019Calibration of Encoder Decoder Models for Neural Machine Translationcited by 67
  3. 2020On the Inference Calibration of Neural Machine Translationcited by 61
  4. 2016Linguistic Input Features Improve Neural Machine Translationcited by 38
  5. 2020Detecting Fine-Grained Cross-Lingual Semantic Divergences without Supervision by Learning to Rankcited by 18
  6. 2020The Sockeye 2 Neural Machine Translation Toolkit at AMTA 2020cited by 16
  7. 2018Identifying Semantic Divergences in Parallel Text without Annotationscited by 9
  8. 2019Novel Applications of Factored Neural Machine Translationcited by 6
  9. 2016Factored Neural Machine Translationcited by 3
  10. 2017Neural Machine Translation by Generating Multiple Linguistic Factorscited by 3
  11. 2019Towards Recognizing Phrase Translation Processes: Experiments on English-Frenchcited by 3

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 4; 2023: 2; 2024: 2.

9 papers.

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  2. 2022Challenges and Strategies in Cross-Cultural NLPcited by 216
  3. 2023Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspectioncited by 44
  4. 2023Cultural Adaptation of Recipescited by 6
  5. 2024How to Learn in a Noisy World? Self-Correcting the Real-World Data Noise in Machine Translationcited by 4
  6. 2021BitextEdit: Automatic Bitext Editing for Improved Low-Resource Machine Translationcited by 3
  7. 2024Learning from Many Voices: Literary MT Using Multi-Reference Human and Synthetic Datacited by 3
  8. 2022Measuring Fine-Grained Semantic Equivalence with Abstract Meaning Representationcited by 2
  9. 2022Can Synthetic Translations Improve Bitext Quality?cited by 1