mlsanityData to 7 Oct 2026

The Curious Case of Hallucinations in Neural Machine Translation

arXiv 2104.06683 ↗First posted 14 Apr 2021cs.CL cs.AI cs.LGcited by 161

Not on the map. Why?

Abstract (arXiv)

In this work, we study hallucinations in Neural Machine Translation (NMT), which lie at an extreme end on the spectrum of NMT pathologies. Firstly, we connect the phenomenon of hallucinations under source perturbation to the Long-Tail theory of Feldman (2020), and present an empirically validated hypothesis that explains hallucinations under source perturbation. Secondly, we consider hallucinations under corpus-level noise (without any source perturbation) and demonstrate that two prominent types of natural hallucinations (detached and oscillatory outputs) could be generated and explained through specific corpus-level noise patterns. Finally, we elucidate the phenomenon of hallucination amplification in popular data-generation processes such as Backtranslation and sequence-level Knowledge Distillation.

Key facts. Cited by 161 papers in the map (data through 7 Oct 2026). Builds on 20 of its 31 references that are in the map. First posted on arXiv 14 Apr 2021.

Builds on

20 of 31 references.

References · 20

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  4. 2016Sequence-Level Knowledge Distillationcited by 761
  5. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  6. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  7. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  8. 2017Six Challenges for Neural Machine Translationcited by 334
  9. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  10. 2016Modeling Coverage for Neural Machine Translationcited by 110
  11. 2019Robust Neural Machine Translation with Doubly Adversarial Inputscited by 105
  12. 2020Tilted Empirical Risk Minimizationcited by 103
8 more
  1. 2020Improved Natural Language Generation via Loss Truncationcited by 75
  2. 2020Analyzing the Source and Target Contributions to Predictions in Neural Machine Translationcited by 71
  3. 2019Domain Robustness in Neural Machine Translationcited by 64
  4. 2018Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corporacited by 40
  5. 2018Filtering and Mining Parallel Data in a Joint Multilingual Spacecited by 32
  6. 2019Naver Labs Europe's Systems for the WMT19 Machine Translation Robustness Taskcited by 22
  7. 2020On Long-Tailed Phenomena in Neural Machine Translationcited by 20
  8. 2020Parallel Corpus Filtering via Pre-trained Language Modelscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 28; 2023: 56; 2024: 33; 2025: 25; 2026: 10.

161 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  2. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  3. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  4. 2023How Language Model Hallucinations Can Snowballcited by 310
  5. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 307
  6. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 260
  7. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 167
  8. 2021Counterfactual Memorization in Neural Language Modelscited by 155
  9. 2022On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?cited by 137
  10. 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 130
  11. 2023Hallucinations in Large Multilingual Translation Modelscited by 126
  12. 2023Almanac: Retrieval-Augmented Language Models for Clinical Medicinecited by 122
38 more
  1. 2022FaithDial: A Faithful Benchmark for Information-Seeking Dialoguecited by 108
  2. 2021Survey of Low-Resource Machine Translationcited by 92
  3. 2023Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkerscited by 86
  4. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  5. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  6. 2021NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentationcited by 72
  7. 2022Understanding Transformer Memorization Recall Through Idiomscited by 62
  8. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 60
  9. 2023Leveraging GPT-4 for Automatic Translation Post-Editingcited by 56
  10. 2023HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attributioncited by 55
  11. 2023PURR: Efficiently Editing Language Model Hallucinations by Denoising Language Model Corruptionscited by 53
  12. 2024CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Modelscited by 50
  13. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 49
  14. 2023Steering Large Language Models for Machine Translation with Finetuning and In-Context Learningcited by 48
  15. 2021The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translationcited by 46
  16. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
  17. 2023Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspectioncited by 44
  18. 2023Effective Large Language Model Adaptation for Improved Grounding and Citation Generationcited by 44
  19. 2023KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detectioncited by 43
  20. 2023Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decodingcited by 42
  21. 2023A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detectioncited by 42
  22. 2023SoK: Memorization in General-Purpose Large Language Modelscited by 36
  23. 2023HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translationcited by 35
  24. 2023Knowledge Solver: Teaching LLMs to Search for Domain Knowledge from Knowledge Graphscited by 35
  25. 2023Learn to Refuse: Making Large Language Models More Controllable and Reliable through Knowledge Scope Limitation and Refusal Mechanismcited by 33
  26. 2024Measuring and Reducing LLM Hallucination without Gold-Standard Answerscited by 31
  27. 2025Investigation of Whisper ASR Hallucinations Induced by Non-Speech Audiocited by 31
  28. 2023Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectivescited by 30
  29. 2023Biomedical knowledge graph-optimized prompt generation for large language modelscited by 30
  30. 2023Elastic Weight Removal for Faithful and Abstractive Dialogue Generationcited by 29
  31. 2023WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpuscited by 29
  32. 2023Improving Translation Faithfulness of Large Language Models via Augmenting Instructionscited by 28
  33. 2024ICT: Image-Object Cross-Level Trusted Intervention for Mitigating Object Hallucination in Large Vision-Language Modelscited by 28
  34. 2022Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methodscited by 27
  35. 2025SemEval-2025 Task 3: Mu-SHROOM, the Multilingual Shared Task on Hallucinations and Related Observable Overgeneration Mistakescited by 27
  36. 2022RuCoLA: Russian Corpus of Linguistic Acceptabilitycited by 26
  37. 2023BLEURT Has Universal Translations: An Analysis of Automatic Metrics by Minimum Risk Trainingcited by 24
  38. 2023Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrievalcited by 24

50 most cited of 161 All 161 (JSON)