mlsanityData to 14 Oct 2026

Hallucinations in Large Multilingual Translation Models

arXiv 2303.16104 ↗First posted 28 Mar 2023cs.CLcited by 129

Not on the map. Why?

Abstract (arXiv)

Large-scale multilingual machine translation systems have demonstrated remarkable ability to translate directly between numerous languages, making them increasingly appealing for real-world applications. However, when deployed in the wild, these models may generate hallucinated translations which have the potential to severely undermine user trust and raise safety concerns. Existing research on hallucinations has primarily focused on small bilingual models trained on high-resource languages, leaving a gap in our understanding of hallucinations in massively multilingual models across diverse translation scenarios. In this work, we fill this gap by conducting a comprehensive analysis on both the M2M family of conventional neural machine translation models and ChatGPT, a general-purpose large language model~(LLM) that can be prompted for translation. Our investigation covers a broad spectrum of conditions, spanning over 100 translation directions across various resource levels and going beyond English-centric language pairs. We provide key insights regarding the prevalence, properties, and mitigation of hallucinations, paving the way towards more responsible and reliable machine translation systems.

Key facts. Cited by 129 papers in the map (data through 14 Oct 2026). Builds on 52 of its 74 references that are in the map. First posted on arXiv 28 Mar 2023.

Builds on

52 of 74 references.

References · 52

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  5. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  7. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  8. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  9. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  10. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  11. 2022Red Teaming Language Models with Language Modelscited by 1,026
  12. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
38 more
  1. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  2. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  3. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  4. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
  5. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  6. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  7. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
  8. 2023GPTScore: Evaluate as You Desirecited by 361
  9. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
  10. 2016FastText.zip: Compressing text classification modelscited by 335
  11. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  12. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  13. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 257
  14. 2019Massively Multilingual Neural Machine Translationcited by 246
  15. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 207
  16. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  17. 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 145
  18. 2023Towards Making the Most of ChatGPT for Machine Translationcited by 138
  19. 2021Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarizationcited by 129
  20. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  21. 2019On NMT Search Errors and Model Errors: Cat Got Your Tongue?cited by 114
  22. 2023The unreasonable effectiveness of few-shot learning for machine translationcited by 91
  23. 2023Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Modelscited by 87
  24. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  25. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  26. 2021NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentationcited by 73
  27. 2022Building Machine Translation Systems for the Next Thousand Languagescited by 69
  28. 2020TICO-19: the Translation Initiative for Covid-19cited by 68
  29. 2019Domain Robustness in Neural Machine Translationcited by 65
  30. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 61
  31. 2023Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOMcited by 59
  32. 2021Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?cited by 54
  33. 2019Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translationcited by 52
  34. 2022Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformercited by 45
  35. 2023Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspectioncited by 44
  36. 2019Naver Labs Europe's Systems for the WMT19 Machine Translation Robustness Taskcited by 23
  37. 2022SALTED: A Framework for SAlient Long-Tail Translation Error Detectioncited by 23
  38. 2022Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learningcited by 19

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 32; 2024: 49; 2025: 36; 2026: 12.

129 papers.

Sort:

By area:

  1. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  2. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 316
  3. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 265
  4. 2023Can LLM-Generated Misinformation Be Detected?cited by 177
  5. 2023Multilingual Machine Translation with Large Language Models: Empirical Results and Analysiscited by 160
  6. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 124
  7. 2024Are self-explanations from Large Language Models faithful?cited by 101
  8. 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 100
  9. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 99
  10. 2024Information Flow Routes: Automatically Interpreting Language Models at Scalecited by 95
  11. 2023Conversational Health Agents: A Personalized LLM-Powered Agent Frameworkcited by 92
  12. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 84
38 more
  1. 2023Large language models effectively leverage document-level context for literary translation, but critical errors persistcited by 80
  2. 2024Understanding and Mitigating Language Confusion in LLMscited by 75
  3. 2024Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedbackcited by 66
  4. 2024CroissantLLM: A Truly Bilingual French-English Language Modelcited by 53
  5. 2024Conformal Prediction for Natural Language Processing: A Surveycited by 53
  6. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  7. 2023Steering Large Language Models for Machine Translation with Finetuning and In-Context Learningcited by 49
  8. 2024Multicalibration for Confidence Scoring in LLMscited by 46
  9. 2025The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating Itcited by 45
  10. 2023Exploring Human-Like Translation Strategy with Large Language Modelscited by 44
  11. 2024Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratingscited by 44
  12. 2023Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decodingcited by 42
  13. 2024Reducing Tool Hallucination via Reliability Alignmentcited by 39
  14. 2024Tool-Planner: Task Planning with Clusters across Multiple Toolscited by 33
  15. 2025Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agentscited by 32
  16. 2024LLMs for Generating and Evaluating Counterfactuals: A Comprehensive Studycited by 31
  17. 2024CaLMQA: Exploring culturally specific long-form question answering across 23 languagescited by 31
  18. 2024Translating Step-by-Step: Decomposing the Translation Process for Improved Translation Quality of Long-Form Textscited by 31
  19. 2023Two Directions for Clinical Data Generation with Large Language Models: Data-to-Label and Label-to-Datacited by 26
  20. 2025Evaluating o1-Like LLMs: Unlocking Reasoning for Translation through Comprehensive Analysiscited by 26
  21. 2024RefChecker: Reference-based Fine-grained Hallucination Checker and Benchmark for Large Language Modelscited by 25
  22. 2024"Not Aligned" is Not "Malicious": Being Careful about Hallucinations of Large Language Models' Jailbreakcited by 25
  23. 2024Boosting Conversational Question Answering with Fine-Grained Retrieval-Augmentation and Self-Checkcited by 24
  24. 2025MASH-VLM: Mitigating Action-Scene Hallucination in Video-LLMs through Disentangled Spatial-Temporal Representationscited by 22
  25. 2023AI-assisted Code Authoring at Scale: Fine-tuning, deploying, and mixed methods evaluationcited by 21
  26. 2024A Probabilistic Framework for LLM Hallucination Detection via Belief Tree Propagationcited by 21
  27. 2025Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Resultscited by 21
  28. 2023Can Large Language Models Provide Security & Privacy Advice? Measuring the Ability of LLMs to Refute Misconceptionscited by 20
  29. 2024Ethos: Rectifying Language Models in Orthogonal Parameter Spacecited by 20
  30. 2025Semantic Volume: Quantifying and Detecting both External and Internal Uncertainty in LLMscited by 20
  31. 2023Simple LLM Prompting is State-of-the-Art for Robust and Multilingual Dialogue Evaluationcited by 19
  32. 2024Cendol: Open Instruction-tuned Generative Large Language Models for Indonesian Languagescited by 18
  33. 2023Online Advertisements with LLMs: Opportunities and Challengescited by 17
  34. 2023TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic Varietiescited by 16
  35. 2023An Empirical Study of Translation Hypothesis Ensembling with Large Language Modelscited by 16
  36. 2024QUEST: Quality-Aware Metropolis-Hastings Sampling for Machine Translationcited by 16
  37. 2024Towards Rationality in Language and Multimodal Agents: A Surveycited by 16
  38. 2025Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missingcited by 14

50 most cited of 129 All 129 (JSON)