mlsanityData to 14 Oct 2026

Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better

arXiv 2212.08597 ↗First posted 16 Dec 2022cs.CLcited by 75

Not on the map. Why?

Abstract (arXiv)

While the problem of hallucinations in neural machine translation has long been recognized, so far the progress on its alleviation is very little. Indeed, recently it turned out that without artificially encouraging models to hallucinate, previously existing methods fall short and even the standard sequence log-probability is more informative. It means that characteristics internal to the model can give much more information than we expect, and before using external models and measures, we first need to ask: how far can we go if we use nothing but the translation model itself ? We propose to use a method that evaluates the percentage of the source contribution to a generated translation. Intuitively, hallucinations are translations "detached" from the source, hence they can be identified by low source contribution. This method improves detection accuracy for the most severe hallucinations by a factor of 2 and is able to alleviate hallucinations at test time on par with the previous best approach that relies on external models. Next, if we move away from internal model characteristics and allow external tools, we show that using sentence similarity from cross-lingual embeddings further improves these results.

Key facts. Cited by 75 papers in the map (data through 14 Oct 2026). Builds on 26 of its 39 references that are in the map. First posted on arXiv 16 Dec 2022.

Builds on

26 of 39 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  5. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  6. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  7. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  8. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  9. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  10. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  11. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 301
  12. 2021To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translationcited by 163
14 more
  1. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  2. 2020Detecting Hallucinated Content in Conditional Neural Sequence Generationcited by 152
  3. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  4. 2020Analyzing the Source and Target Contributions to Predictions in Neural Machine Translationcited by 71
  5. 2019Domain Robustness in Neural Machine Translationcited by 65
  6. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 61
  7. 2016A Simple, Fast Diverse Decoding Algorithm for Neural Generationcited by 58
  8. 2020Unbabel's Participation in the WMT20 Metrics Shared Taskcited by 56
  9. 2021Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translationcited by 56
  10. 2022Quality-Aware Decoding for Neural Machine Translationcited by 49
  11. 2022Bitext Mining Using Distilled Sentence Representations for Low-Resource Languagescited by 48
  12. 2022Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformercited by 45
  13. 2019Naver Labs Europe's Systems for the WMT19 Machine Translation Robustness Taskcited by 23
  14. 2022SALTED: A Framework for SAlient Long-Tail Translation Error Detectioncited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 16; 2024: 33; 2025: 19; 2026: 6.

75 papers.

Sort:

By area:

  1. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  2. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  3. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 162
  4. 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 131
  5. 2023Hallucinations in Large Multilingual Translation Modelscited by 129
  6. 2024Are self-explanations from Large Language Models faithful?cited by 101
  7. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 99
  8. 2024Information Flow Routes: Automatically Interpreting Language Models at Scalecited by 95
  9. 2023Alleviating Hallucinations of Large Language Models through Induced Hallucinationscited by 79
  10. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  11. 2023On Early Detection of Hallucinations in Factual Question Answeringcited by 56
  12. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 49
38 more
  1. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 46
  2. 2024Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Modelscited by 44
  3. 2023Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decodingcited by 42
  4. 2023HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translationcited by 35
  5. 2024Mitigating Large Language Model Hallucination with Faithful Finetuningcited by 24
  6. 2023A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translationcited by 19
  7. 2024Comparing Hallucination Detection Metrics for Multilingual Generationcited by 18
  8. 2024Know When To Stop: A Study of Semantic Drift in Text Generationcited by 18
  9. 2024An Audit on the Perspectives and Challenges of Hallucinations in NLPcited by 17
  10. 2024xTower: A Multilingual LLM for Explaining and Correcting Translation Errorscited by 17
  11. 2024LRP4RAG: Detecting Hallucinations in Retrieval-Augmented Generation via Layer-wise Relevance Propagationcited by 17
  12. 2023Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarizationcited by 16
  13. 2024Word Alignment as Preference for Machine Translationcited by 14
  14. 2024Teaching LLMs to Abstain across Languages via Multilingual Feedbackcited by 14
  15. 2022Optimal Transport for Unsupervised Hallucination Detection in Neural Machine Translationcited by 13
  16. 2024Machine Translation Hallucination Detection for Low and High Resource Languages using Large Language Modelscited by 13
  17. 2024Understanding Reasoning in Chain-of-Thought from the Hopfieldian Viewcited by 11
  18. 2025Uncertainty-Aware Decoding with Minimum Bayes Riskcited by 11
  19. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  20. 2024On Measuring Context Utilization in Document-Level MT Systemscited by 9
  21. 2024Enhanced Hallucination Detection in Neural Machine Translation through Simple Detector Aggregationcited by 9
  22. 2024Detecting Hallucination and Coverage Errors in Retrieval Augmented Generation for Controversial Topicscited by 9
  23. 2024Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunitiescited by 8
  24. 2025Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translationcited by 8
  25. 2023Quantifying the Plausibility of Context Reliance in Neural Machine Translationcited by 7
  26. 2023Hallucination Detection for Grounded Instruction Generationcited by 7
  27. 2024A Survey of Automatic Hallucination Evaluation on Natural Language Generationcited by 7
  28. 2024How Effective are State Space Models for Machine Translation?cited by 6
  29. 2025Mitigating Hallucinated Translations in Large Language Models with Hallucination-focused Preference Optimizationcited by 6
  30. 2025QE4PE: Word-level Quality Estimation for Human Post-Editingcited by 6
  31. 2024The Pitfalls of Defining Hallucinationcited by 5
  32. 2025Two Intermediate Translations Are Better Than One: Fine-tuning LLMs for Document-level Translation Refinementcited by 5
  33. 2025Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreementcited by 4
  34. 2025Hallucinations in Code Change to Natural Language Generation: Prevalence and Evaluation of Detection Metricscited by 4
  35. 2025M2PO: Multi-Perspective Multi-Pair Preference Optimization for Machine Translationcited by 4
  36. 2023Progressive Translation: Improving Domain Robustness of Neural Machine Translation with Intermediate Sequencescited by 3
  37. 2023ReSeTOX: Re-learning attention weights for toxicity mitigation in machine translationcited by 3
  38. 2024Towards Red Teaming in Multimodal and Multilingual Translationcited by 3

50 most cited of 75 All 75 (JSON)