# Facts Do Care About Your Language: Assessing Answer Quality of Multilingual LLMs

arXiv: https://arxiv.org/abs/2506.03051 · HTML: https://mlsanity.com/paper/2506.03051/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 11 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Builds on (most cited first, 6 of 6)

- [Document-Level Machine Translation with Large Language Models](https://mlsanity.com/paper/2304.02210/) (2023)
- [Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkers](https://mlsanity.com/paper/2311.09000/) (2023)
- [Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries](https://mlsanity.com/paper/2310.13132/) (2023)
- [On the Diversity of Synthetic Data and its Impact on Training Large Language Models](https://mlsanity.com/paper/2410.15226/) (2024)
- [LLM-powered Data Augmentation for Enhanced Cross-lingual Performance](https://mlsanity.com/paper/2305.14288/) (2023)
- [Evaluating and Mitigating Linguistic Discrimination in Large Language Models](https://mlsanity.com/paper/2404.18534/) (2024)

## Abstract (arXiv)

Factuality is a necessary precursor to useful educational tools. As adoption of Large Language Models (LLMs) in education continues of grow, ensuring correctness in all settings is paramount. Despite their strong English capabilities, LLM performance in other languages is largely untested. In this work, we evaluate the correctness of the Llama3.1 family of models in answering factual questions appropriate for middle and high school students. We demonstrate that LLMs not only provide extraneous and less truthful information, but also exacerbate existing biases against rare languages.