# Facts are Harder Than Opinions -- A Multilingual, Comparative Analysis of LLM-Based Fact-Checking Reliability

arXiv: https://arxiv.org/abs/2506.03655 · HTML: https://mlsanity.com/paper/2506.03655/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 52 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Builds on (most cited first, 28 of 28)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023)
- [Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence](https://mlsanity.com/paper/2103.08541/) (2021)
- [Demystifying Prompts in Language Models via Perplexity Estimation](https://mlsanity.com/paper/2212.04037/) (2022)
- [Explainable Automated Fact-Checking for Public Health Claims](https://mlsanity.com/paper/2010.09926/) (2020)
- [MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims](https://mlsanity.com/paper/1909.03242/) (2019)
- [Towards Debiasing Fact Verification Models](https://mlsanity.com/paper/1908.05267/) (2019)
- [DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning](https://mlsanity.com/paper/1809.06416/) (2018)
- [X-FACT: A New Benchmark Dataset for Multilingual Fact Checking](https://mlsanity.com/paper/2106.09248/) (2021)
- [Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance](https://mlsanity.com/paper/2005.00315/) (2020)
- [Explainable Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2011.03870/) (2020)
- [A Richly Annotated Corpus for Different Tasks in Automated Fact-Checking](https://mlsanity.com/paper/1911.01214/) (2019)
- [Language Models as Fact Checkers?](https://mlsanity.com/paper/2006.04102/) (2020)
- [Fool Me Twice: Entailment from Wikipedia Gamification](https://mlsanity.com/paper/2104.04725/) (2021)
- [Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4](https://mlsanity.com/paper/2305.14928/) (2023)
- [The language of prompting: What linguistic properties make a prompt successful?](https://mlsanity.com/paper/2311.01967/) (2023)
- [FactLLaMA: Optimizing Instruction-Following Language Models with External Knowledge for Automated Fact-Checking](https://mlsanity.com/paper/2309.00240/) (2023)
- [Large Language Models Offer an Alternative to the Traditional Approach of Topic Modelling](https://mlsanity.com/paper/2403.16248/) (2024)
- [Fine-grained Affective Processing Capabilities Emerging from Large Language Models](https://mlsanity.com/paper/2309.01664/) (2023)
- [From alternative conceptions of honesty to alternative facts in communications by U.S. politicians](https://mlsanity.com/paper/2208.10814/) (2022)
- [The Half-Life of a Tweet](https://mlsanity.com/paper/2302.09654/) (2023)
- [GenAI vs. Human Fact-Checkers: Accurate Ratings, Flawed Rationales](https://mlsanity.com/paper/2502.14943/) (2025)

## Built on by (most cited first, 3 of 3)

- [Rubric-Conditioned LLM Grading: Alignment, Uncertainty, and Robustness](https://mlsanity.com/paper/2601.08843/) (2025)
- [Paladin-mini: A Compact and Efficient Grounding Model Excelling in Real-World Scenarios](https://mlsanity.com/paper/2506.20384/) (2025)
- [Enhancing Debunking Effectiveness through LLM-based Personality Adaptation](https://mlsanity.com/paper/2603.09533/) (2026)

## Abstract (arXiv)

The proliferation of misinformation necessitates scalable, automated fact-checking solutions. Yet, current benchmarks often overlook multilingual and topical diversity. This paper introduces a novel, dynamically extensible data set that includes 61,514 claims in multiple languages and topics, extending existing datasets up to 2024. Through a comprehensive evaluation of five prominent Large Language Models (LLMs), including GPT-4o, GPT-3.5 Turbo, LLaMA 3.1, and Mixtral 8x7B, we identify significant performance gaps between different languages and topics. While overall GPT-4o achieves the highest accuracy, it declines to classify 43% of claims. Across all models, factual-sounding claims are misclassified more often than opinions, revealing a key vulnerability. These findings underscore the need for caution and highlight challenges in deploying LLM-based fact-checking systems at scale.