# A Comparative Study of Translation Bias and Accuracy in Multilingual Large Language Models for Cross-Language Claim Verification

arXiv: https://arxiv.org/abs/2410.10303 · HTML: https://mlsanity.com/paper/2410.10303/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 27 references that are in the map.
- First posted on arXiv 14 Oct 2024.

## Builds on (most cited first, 20 of 20)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task](https://mlsanity.com/paper/2209.06243/) (2022)
- [Automated Fact Checking: Task formulations, methods and future directions](https://mlsanity.com/paper/1806.07687/) (2018)
- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [X-FACT: A New Benchmark Dataset for Multilingual Fact Checking](https://mlsanity.com/paper/2106.09248/) (2021)
- [Do Multilingual Language Models Think Better in English?](https://mlsanity.com/paper/2308.01223/) (2023)
- [A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias](https://mlsanity.com/paper/2404.00929/) (2024)
- [Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOM](https://mlsanity.com/paper/2303.01911/) (2023)
- [MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks](https://mlsanity.com/paper/2311.07463/) (2023)
- [Do Large Language Models Know about Facts?](https://mlsanity.com/paper/2310.05177/) (2023)
- [A Survey on Large Language Models with Multilingualism: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.10936/) (2024)
- [Claim Detection for Automated Fact-checking: A Survey on Monolingual, Multilingual and Cross-Lingual Research](https://mlsanity.com/paper/2401.11969/) (2024)
- [The Perils & Promises of Fact-checking with Large Language Models](https://mlsanity.com/paper/2310.13549/) (2023)
- [ChatGPT MT: Competitive for High- (but not Low-) Resource Languages](https://mlsanity.com/paper/2309.07423/) (2023)
- [On the Multilingual Capabilities of Very Large-Scale English Language Models](https://mlsanity.com/paper/2108.13349/) (2021)
- [From Chaos to Clarity: Claim Normalization to Empower Fact-Checking](https://mlsanity.com/paper/2310.14338/) (2023)
- [Do We Need Language-Specific Fact-Checking Models? The Case of Chinese](https://mlsanity.com/paper/2401.15498/) (2024)
- [CONCRETE: Improving Cross-lingual Fact-checking with Cross-lingual Retrieval](https://mlsanity.com/paper/2209.02071/) (2022)
- [Breaking the Language Barrier: Can Direct Inference Outperform Pre-Translation in Multilingual LLM Applications?](https://mlsanity.com/paper/2403.04792/) (2024)
- [Evaluating and Mitigating Linguistic Discrimination in Large Language Models](https://mlsanity.com/paper/2404.18534/) (2024)

## Built on by (most cited first, 1 of 1)

- [Fair Fact-Checking: Closing the Cross-Lingual Gap in LLM Factual Judgement with RoSh](https://mlsanity.com/paper/2609.34678/) (2026)

## Abstract (arXiv)

The rise of digital misinformation has heightened interest in using multilingual Large Language Models (LLMs) for fact-checking. This study systematically evaluates translation bias and the effectiveness of LLMs for cross-lingual claim verification across 15 languages from five language families: Romance, Slavic, Turkic, Indo-Aryan, and Kartvelian. Using the XFACT dataset to assess their impact on accuracy and bias, we investigate two distinct translation methods: pre-translation and self-translation. We use mBERT's performance on the English dataset as a baseline to compare language-specific accuracies. Our findings reveal that low-resource languages exhibit significantly lower accuracy in direct inference due to underrepresentation in the training data. Furthermore, larger models demonstrate superior performance in self-translation, improving translation accuracy and reducing bias. These results highlight the need for balanced multilingual training, especially in low-resource languages, to promote equitable access to reliable fact-checking tools and minimize the risk of spreading misinformation in different linguistic contexts.