mlsanityData to 7 Oct 2026

On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation

arXiv 2005.03642 ↗First posted 7 May 2020cs.CLcited by 127

Not on the map. Why?

Abstract (arXiv)

The standard training algorithm in neural machine translation (NMT) suffers from exposure bias, and alternative algorithms have been proposed to mitigate this. However, the practical impact of exposure bias is under debate. In this paper, we link exposure bias to another well-known problem in NMT, namely the tendency to generate hallucinations under domain shift. In experiments on three datasets with multiple test domains, we show that exposure bias is partially to blame for hallucinations, and that training with Minimum Risk Training, which avoids exposure bias, can mitigate this. Our analysis explains why exposure bias is more problematic under domain shift, and also links exposure bias to the beam search problem, i.e. performance deterioration with increasing beam size. Our results provide a new justification for methods that reduce exposure bias: even if they do not increase performance on in-domain test sets, they can increase model robustness to domain shift.

Key facts. Cited by 127 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 28; 2022: 23; 2023: 29; 2024: 20; 2025: 12; 2026: 14.

127 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  3. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  4. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  5. 2023How Language Model Hallucinations Can Snowballcited by 310
  6. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  7. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  8. 2021A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generationcited by 137
  9. 2022On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?cited by 137
  10. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 131
  11. 2021Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarizationcited by 128
  12. 2023Hallucinations in Large Multilingual Translation Modelscited by 126
38 more
  1. 2022FaithDial: A Faithful Benchmark for Information-Seeking Dialoguecited by 108
  2. 2021Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Groundingcited by 106
  3. 2022Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generationcited by 106
  4. 2022ChatGPT Makes Medicine Easy to Swallow: An Exploratory Case Study on Simplified Radiology Reportscited by 100
  5. 2021Survey of Low-Resource Machine Translationcited by 92
  6. 2021Do Long-Range Language Models Actually Use Long-Range Context?cited by 92
  7. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  8. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  9. 2023Automatic Evaluation of Attribution by Large Language Modelscited by 70
  10. 2024Learning Harmonized Representations for Speculative Samplingcited by 69
  11. 2022Mutual Information Alleviates Hallucinations in Abstractive Summarizationcited by 67
  12. 2020Text Generation by Learning from Demonstrationscited by 64
  13. 2021Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translationcited by 63
  14. 2024We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMscited by 61
  15. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 60
  16. 2022Quality-Aware Decoding for Neural Machine Translationcited by 49
  17. 2021Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Surveycited by 46
  18. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
  19. 2023Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspectioncited by 44
  20. 2022Contrastive Learning Reduces Hallucination in Conversationscited by 39
  21. 2025RLVR-World: Training World Models with Reinforcement Learningcited by 38
  22. 2023SoK: Memorization in General-Purpose Large Language Modelscited by 36
  23. 2021Prevent the Language Model from being Overconfident in Neural Machine Translationcited by 34
  24. 2021Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translationcited by 32
  25. 2023Elastic Weight Removal for Faithful and Abstractive Dialogue Generationcited by 29
  26. 2022Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methodscited by 27
  27. 2023BLEURT Has Universal Translations: An Analysis of Automatic Metrics by Minimum Risk Trainingcited by 24
  28. 2023Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreementcited by 22
  29. 2021Sketch and Refine: Towards Faithful and Informative Table-to-Text Generationcited by 21
  30. 2024Language Models in Dialogue: Conversational Maxims for Human-AI Interactionscited by 21
  31. 2024Better than Your Teacher: LLM Agents that learn from Privileged AI Feedbackcited by 21
  32. 2021Relating Neural Text Degeneration to Exposure Biascited by 19
  33. 2023Generative Models as a Complex Systems Science: How can we make sense of large language model behavior?cited by 19
  34. 2023Zero-Resource Hallucination Prevention for Large Language Modelscited by 19
  35. 2024The Accuracy Paradox in RLHF: When Better Reward Models Don't Yield Better Language Modelscited by 19
  36. 2025Too Consistent to Detect: A Study of Self-Consistent Errors in LLMscited by 18
  37. 2024An Audit on the Perspectives and Challenges of Hallucinations in NLPcited by 17
  38. 2021Modelling Latent Translations for Cross-Lingual Transfercited by 16

50 most cited of 127 All 127 (JSON)