mlsanityData to 14 Oct 2026

Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection

arXiv 2301.07779 ↗First posted 18 Jan 2023cs.CLcited by 44

Not on the map. Why?

Abstract (arXiv)

Neural sequence generation models are known to "hallucinate", by producing outputs that are unrelated to the source text. These hallucinations are potentially harmful, yet it remains unclear in what conditions they arise and how to mitigate their impact. In this work, we first identify internal model symptoms of hallucinations by analyzing the relative token contributions to the generation in contrastive hallucinated vs. non-hallucinated outputs generated via source perturbations. We then show that these symptoms are reliable indicators of natural hallucinations, by using them to design a lightweight hallucination detector which outperforms both model-free baselines and strong classifiers based on quality estimation or large pre-trained models on manually annotated English-Chinese and German-English translation test beds.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). Builds on 43 of its 75 references that are in the map. First posted on arXiv 18 Jan 2023.

Builds on

43 of 75 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  5. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  7. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  8. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  9. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  10. 2020FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarizationcited by 293
  11. 2017Challenges in Data-to-Document Generationcited by 226
  12. 2016Understanding Neural Networks through Representation Erasurecited by 224
31 more
  1. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 223
  2. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 211
  3. 2015Visualizing and Understanding Neural Models in NLPcited by 200
  4. 2016Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layerscited by 199
  5. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  6. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  7. 2020Detecting Hallucinated Content in Conditional Neural Sequence Generationcited by 152
  8. 2018Pathologies of Neural Models Make Interpretations Difficultcited by 140
  9. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  10. 2018Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Contextcited by 113
  11. 2018On Adversarial Examples for Character-Level Neural Machine Translationcited by 85
  12. 2020Analyzing the Source and Target Contributions to Predictions in Neural Machine Translationcited by 71
  13. 2019The FLoRes Evaluation Datasets for Low-Resource Machine Translation: Nepali-English and Sinhala-Englishcited by 62
  14. 2020On the Inference Calibration of Neural Machine Translationcited by 61
  15. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 61
  16. 2020TransQuest: Translation Quality Estimation with Cross-lingual Transformerscited by 59
  17. 2017A causal framework for explaining the predictions of black-box sequence-to-sequence modelscited by 58
  18. 2020Unbabel's Participation in the WMT20 Metrics Shared Taskcited by 56
  19. 2019Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translationcited by 52
  20. 2022Bitext Mining Using Distilled Sentence Representations for Low-Resource Languagescited by 48
  21. 2021Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generationcited by 29
  22. 2018Findings of the E2E NLG Challengecited by 28
  23. 2019Saliency-driven Word Alignment Interpretation for Neural Machine Translationcited by 27
  24. 2019Naver Labs Europe's Systems for the WMT19 Machine Translation Robustness Taskcited by 23
  25. 2018Fluency Over Adequacy: A Pilot Study in Measuring User Trust in Imperfect MTcited by 21
  26. 2019Towards Understanding Neural Machine Translation with Word Importancecited by 19
  27. 2020Detecting Fine-Grained Cross-Lingual Semantic Divergences without Supervision by Learning to Rankcited by 18
  28. 2019Machine Translation of Restaurant Reviews: New Corpus for Domain Adaptation and Robustnesscited by 10
  29. 2021Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translationcited by 9
  30. 2017Confidence through Attentioncited by 8
  31. 2022Rare but Severe Neural Machine Translation Errors Induced by Minimal Deletion: An Empirical Study on Chinese and Englishcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 14; 2024: 17; 2025: 9; 2026: 4.

44 papers.

Sort:

By area:

  1. 2023Fine-tuning Language Models for Factualitycited by 221
  2. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 150
  3. 2023LM-Polygraph: Uncertainty Estimation for Language Modelscited by 131
  4. 2023Hallucinations in Large Multilingual Translation Modelscited by 129
  5. 2024A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthinesscited by 127
  6. 2023Calibrated Language Models Must Hallucinatecited by 115
  7. 2024Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraphcited by 105
  8. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 99
  9. 2024Unfamiliar Finetuning Examples Control How Language Models Hallucinatecited by 87
  10. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  11. 2023Do Androids Know They're Only Dreaming of Electric Sheep?cited by 67
  12. 2023Leveraging GPT-4 for Automatic Translation Post-Editingcited by 56
32 more
  1. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 46
  2. 2023HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translationcited by 35
  3. 2024Generative AI for Synthetic Data Generation: Methods, Challenges and the Futurecited by 33
  4. 2024Towards Detecting LLMs Hallucination via Markov Chain-based Multi-agent Debate Frameworkcited by 19
  5. 2024Comparing Hallucination Detection Metrics for Multilingual Generationcited by 18
  6. 2023An Empirical Study of Translation Hypothesis Ensembling with Large Language Modelscited by 16
  7. 2023Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarizationcited by 16
  8. 2024Teaching LLMs to Abstain across Languages via Multilingual Feedbackcited by 14
  9. 2024Machine Translation Hallucination Detection for Low and High Resource Languages using Large Language Modelscited by 13
  10. 2024DHCP: Detecting Hallucinations by Cross-modal Attention Pattern in Large Vision-Language Modelscited by 11
  11. 2024Structured Chain-of-Thought Prompting for Few-Shot Generation of Content-Grounded QA Conversationscited by 9
  12. 2023Hallucination Detection for Grounded Instruction Generationcited by 7
  13. 2024Low-resource neural machine translation with morphological modelingcited by 7
  14. 2025QE4PE: Word-level Quality Estimation for Human Post-Editingcited by 6
  15. 2025Neural Message-Passing on Attention Graphs for Hallucination Detectioncited by 6
  16. 2025A Unified Definition of Hallucination: It's The World Model, Stupid!cited by 6
  17. 2023Think While You Write: Hypothesis Verification Promotes Faithful Knowledge-to-Text Generationcited by 5
  18. 2025Controlling Large Language Model with Latent Actionscited by 5
  19. 2026HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Auto-Encoderscited by 5
  20. 2026Trust Me, Import This: Dependency Steering Attacks via Malicious Agent Skillscited by 5
  21. 2025Multi-Agent Actor-Critic Generative AI for Query Resolution and Analysiscited by 3
  22. 2024Successfully Guiding Humans with Imperfect Instructions by Highlighting Potential Errors and Suggesting Correctionscited by 2
  23. 2024Evaluating the Accuracy of Chatbots in Financial Literaturecited by 2
  24. 2024What Have We Achieved on Non-autoregressive Translation?cited by 1
  25. 2025Toward Machine Translation Literacy: How Lay Users Perceive and Rely on Imperfect Translationscited by 1
  26. 2025SymLoc: Symbolic Localization of Hallucination across HaluEval and TruthfulQAcited by 1
  27. 2026Beyond In-Domain Detection: SpikeScore for Cross-Domain Hallucination Detectioncited by 1
  28. 2026Pre-Generation Hallucination Detection in Large Language Models via Soft-Target Attention Probingcited by 1
  29. 2023Two Approaches to Supervised Image Segmentationcited by 0
  30. 2024Non-Fluent Synthetic Target-Language Data Improve Neural Machine Translationcited by 0
  31. 2025MMD-Flagger: Leveraging Maximum Mean Discrepancy to Detect Hallucinationscited by 0
  32. 2025Multi-Hypothesis Distillation of Multilingual Neural Translation Models for Low-Resource Languagescited by 0