# Investigating Language and Retrieval Bias in Multilingual Previously Fact-Checked Claim Detection

arXiv: https://arxiv.org/abs/2509.25138 · HTML: https://mlsanity.com/paper/2509.25138/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 29 Sep 2025.

## Builds on (most cited first, 18 of 18)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Large Dual Encoders Are Generalizable Retrievers](https://mlsanity.com/paper/2112.07899/) (2021)
- [A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios](https://mlsanity.com/paper/2010.12309/) (2020)
- [Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting](https://mlsanity.com/paper/2305.07004/) (2023)
- [A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias](https://mlsanity.com/paper/2404.00929/) (2024)
- [That is a Known Lie: Detecting Previously Fact-Checked Claims](https://mlsanity.com/paper/2005.06058/) (2020)
- [A Survey on Popularity Bias in Recommender Systems](https://mlsanity.com/paper/2308.01118/) (2023)
- [Claim Matching Beyond English to Scale Global Fact-Checking](https://mlsanity.com/paper/2106.00853/) (2021)
- [User-centered Evaluation of Popularity Bias in Recommender Systems](https://mlsanity.com/paper/2103.06364/) (2021)
- [Multilingual Previously Fact-Checked Claim Retrieval](https://mlsanity.com/paper/2305.07991/) (2023)
- [Assisting the Human Fact-Checkers: Detecting All Previously Fact-Checked Claims in a Document](https://mlsanity.com/paper/2109.07410/) (2021)
- [Multilingual Graphemic Hybrid ASR with Massive Data Augmentation](https://mlsanity.com/paper/1909.06522/) (2019)
- [Zero-shot and Few-shot Learning with Instruction-following LLMs for Claim Matching in Automated Fact-checking](https://mlsanity.com/paper/2501.10860/) (2025)
- [SemEval-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval](https://mlsanity.com/paper/2505.10740/) (2025)
- [Large Language Models for Multilingual Previously Fact-Checked Claim Detection](https://mlsanity.com/paper/2503.02737/) (2025)

## Built on by (most cited first, 1 of 1)

- [Latent Causal Void: Explicit Missing-Context Reconstruction for Misinformation Detection](https://mlsanity.com/paper/2605.12156/) (2026)

## Abstract (arXiv)

Multilingual Large Language Models (LLMs) offer powerful capabilities for cross-lingual fact-checking. However, these models often exhibit language bias, performing disproportionately better on high-resource languages such as English than on low-resource counterparts. We also present and inspect a novel concept - retrieval bias, when information retrieval systems tend to favor certain information over others, leaving the retrieval process skewed. In this paper, we study language and retrieval bias in the context of Previously Fact-Checked Claim Detection (PFCD). We evaluate six open-source multilingual LLMs across 20 languages using a fully multilingual prompting strategy, leveraging the AMC-16K dataset. By translating task prompts into each language, we uncover disparities in monolingual and cross-lingual performance and identify key trends based on model family, size, and prompting strategy. Our findings highlight persistent bias in LLM behavior and offer recommendations for improving equity in multilingual fact-checking. To investigate retrieval bias, we employed multilingual embedding models and look into the frequency of retrieved claims. Our analysis reveals that certain claims are retrieved disproportionately across different posts, leading to inflated retrieval performance for popular claims while under-representing less common ones.