mlsanityData to 14 Oct 2026

Optimal Transport for Unsupervised Hallucination Detection in Neural Machine Translation

arXiv 2212.09631 ↗First posted 19 Dec 2022cs.CL cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Neural machine translation (NMT) has become the de-facto standard in real-world machine translation applications. However, NMT models can unpredictably produce severely pathological translations, known as hallucinations, that seriously undermine user trust. It becomes thus crucial to implement effective preventive strategies to guarantee their proper functioning. In this paper, we address the problem of hallucination detection in NMT by following a simple intuition: as hallucinations are detached from the source content, they exhibit encoder-decoder attention patterns that are statistically different from those of good quality translations. We frame this problem with an optimal transport formulation and propose a fully unsupervised, plug-in detector that can be used with any attention-based NMT model. Experimental results show that our detector not only outperforms all previous model-based detectors, but is also competitive with detectors that employ large models trained on millions of samples.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 30 of its 49 references that are in the map. First posted on arXiv 19 Dec 2022.

Builds on

30 of 49 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  6. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  7. 2022Red Teaming Language Models with Language Modelscited by 1,026
  8. 2018Computational Optimal Transportcited by 780
  9. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  10. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 207
  11. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  12. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
18 more
  1. 2020Detecting Hallucinated Content in Conditional Neural Sequence Generationcited by 152
  2. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  3. 2019Subspace Robust Wasserstein Distancescited by 90
  4. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  5. 2020Analyzing the Source and Target Contributions to Predictions in Neural Machine Translationcited by 71
  6. 2019Domain Robustness in Neural Machine Translationcited by 65
  7. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 61
  8. 2020Unbabel's Participation in the WMT20 Metrics Shared Taskcited by 56
  9. 2021Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translationcited by 56
  10. 2020MLQE-PE: A Multilingual Quality Estimation and Post-Editing Datasetcited by 47
  11. 2022Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformercited by 45
  12. 2019Naver Labs Europe's Systems for the WMT19 Machine Translation Robustness Taskcited by 23
  13. 2020When OT meets MoM: Robust estimation of Wasserstein Distancecited by 23
  14. 2022SALTED: A Framework for SAlient Long-Tail Translation Error Detectioncited by 23
  15. 2021Attention Weights in Transformer NMT Fail Aligning Words Between Sequences but Largely Explain Model Predictionscited by 11
  16. 2021WOOD: Wasserstein-based Out-of-Distribution Detectioncited by 9
  17. 2022Reducing Hallucinations in Neural Machine Translation with Feature Attributioncited by 5
  18. 20192D Wasserstein Loss for Robust Facial Landmark Detectioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 3; 2025: 2; 2026: 2.

13 papers.

Sort:

By area:

  1. 2023Hallucinations in Large Multilingual Translation Modelscited by 129
  2. 2024Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMscited by 63
  3. 2023Leveraging GPT-4 for Automatic Translation Post-Editingcited by 56
  4. 2023HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translationcited by 35
  5. 2023Elastic Weight Removal for Faithful and Abstractive Dialogue Generationcited by 29
  6. 2023Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarizationcited by 16
  7. 2023Unsupervised Layer-wise Score Aggregation for Textual OOD Detectioncited by 15
  8. 2025Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Reviewcited by 4
  9. 2024Non-Fluent Synthetic Target-Language Data Improve Neural Machine Translationcited by 0
  10. 2024Randomized Transport Plans via Hierarchical Fully Probabilistic Designcited by 0
  11. 2025MMD-Flagger: Leveraging Maximum Mean Discrepancy to Detect Hallucinationscited by 0
  12. 2026Self-Attention as Transport: Limits of Symmetric Spectral Diagnosticscited by 0
1 more
  1. 2026Layer-Resolved Optimal Transport for Hallucination Detection in NMT and Abstractive Summarizationcited by 0