# Machine Translation Hallucination Detection for Low and High Resource Languages using Large Language Models

arXiv: https://arxiv.org/abs/2407.16470 · HTML: https://mlsanity.com/paper/2407.16470/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 23 Jul 2024.

## Builds on (most cited first, 15 of 15)

- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection](https://mlsanity.com/paper/2310.10482/) (2023)
- [Better Zero-Shot Reasoning with Role-Play Prompting](https://mlsanity.com/paper/2308.07702/) (2023)
- [MADLAD-400: A Multilingual And Document-Level Large Audited Dataset](https://mlsanity.com/paper/2309.04662/) (2023)
- [SONAR: Sentence-Level Multimodal and Language-Agnostic Representations](https://mlsanity.com/paper/2308.11466/) (2023)
- [Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus](https://mlsanity.com/paper/2311.13230/) (2023)
- [The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation](https://mlsanity.com/paper/2308.07286/) (2023)
- [Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?](https://mlsanity.com/paper/2309.07462/) (2023)
- [Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better](https://mlsanity.com/paper/2212.08597/) (2022)
- [Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection](https://mlsanity.com/paper/2301.07779/) (2023)
- [HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translation](https://mlsanity.com/paper/2305.11746/) (2023)
- [SALTED: A Framework for SAlient Long-Tail Translation Error Detection](https://mlsanity.com/paper/2205.09988/) (2022)

## Built on by (most cited first, 13 of 13)

- [CCHall: A Novel Benchmark for Joint Cross-Lingual and Cross-Modal Hallucinations Detection in Large Language Models](https://mlsanity.com/paper/2505.19108/) (2025)
- [HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Auto-Encoders](https://mlsanity.com/paper/2604.16430/) (2026)
- [Designing Domain-Specific Large Language Models: The Critical Role of Fine-Tuning in Public Opinion Simulation](https://mlsanity.com/paper/2409.19308/) (2024)
- [Generating bilingual example sentences with large language models as lexicography assistants](https://mlsanity.com/paper/2410.03182/) (2024)
- [Turk-LettuceDetect: A Hallucination Detection Models for Turkish RAG Applications](https://mlsanity.com/paper/2509.17671/) (2025)
- [Analyzing Context Contributions in LLM-based Machine Translation](https://mlsanity.com/paper/2410.16246/) (2024)
- [Leveraging NTPs for Efficient Hallucination Detection in VLMs](https://mlsanity.com/paper/2509.20379/) (2025)
- [MMD-Flagger: Leveraging Maximum Mean Discrepancy to Detect Hallucinations](https://mlsanity.com/paper/2506.01367/) (2025)
- [Analyzing Information-Seeking Behaviors in a Hakka AI Chatbot: A Cognitive-Pragmatic Study](https://mlsanity.com/paper/2509.11591/) (2025)
- [On Non-interactive Evaluation of Animal Communication Translators](https://mlsanity.com/paper/2510.15768/) (2025)
- [A Critical Study of Automatic Evaluation in Sign Language Translation](https://mlsanity.com/paper/2510.25434/) (2025)
- [Augmenting Question Answering with A Hybrid RAG Approach](https://mlsanity.com/paper/2601.12658/) (2026)
- [Fabricator or dynamic translator?](https://mlsanity.com/paper/2604.15165/) (2026)

## Abstract (arXiv)

Recent advancements in massively multilingual machine translation systems have significantly enhanced translation accuracy; however, even the best performing systems still generate hallucinations, severely impacting user trust. Detecting hallucinations in Machine Translation (MT) remains a critical challenge, particularly since existing methods excel with High-Resource Languages (HRLs) but exhibit substantial limitations when applied to Low-Resource Languages (LRLs). This paper evaluates sentence-level hallucination detection approaches using Large Language Models (LLMs) and semantic similarity within massively multilingual embeddings. Our study spans 16 language directions, covering HRLs, LRLs, with diverse scripts. We find that the choice of model is essential for performance. On average, for HRLs, Llama3-70B outperforms the previous state of the art by as much as 0.16 MCC (Matthews Correlation Coefficient). However, for LRLs we observe that Claude Sonnet outperforms other LLMs on average by 0.03 MCC. The key takeaway from our study is that LLMs can achieve performance comparable or even better than previously proposed models, despite not being explicitly trained for any machine translation task. However, their advantage is less significant for LRLs.