mlsanityData to 14 Oct 2026

Fair Fact-Checking: Closing the Cross-Lingual Gap in LLM Factual Judgement with RoSh

arXiv 2609.34678 ↗First posted 28 Sep 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Misinformation on social media remains a critical problem, and more and more people settle it by asking a language model instead of a fact checker. Whether models judge such claims reliably is debated; whether they judge them equally well in every language people ask in has gone almost unasked. We test eight models from five families, 3B to 70B, on 1,500 encyclopedic factual claims that exist in identical form in eight languages. English is judged better than every other language on every model, and the gap is widest on the smallest ones, where Llama-3B on Arabic is no better than guessing. Existing remedies retrain on more multilingual data or fit an unconstrained map between language representations, and neither asks whether the model already holds the answer and simply fails to say it. It largely does: a linear probe recovers the truth from the very activations the model fails to express. We propose RoSh, a per-language shift and rotation of the residual stream, computed in closed form at three layers, with no training and no weight modified. It improves every model and closes 75% of the gap on average, helping most where the model was worst: Arabic on Llama-3B goes from chance to nearly the English level, and a fifth fewer of the claims answered correctly in English are lost in translation. What remains is no longer a read-out failure: afterwards the head recovers as much of what is encoded outside English as it does in English. An unconstrained map fitted on the same pairs falls below the untouched baseline, so the orthogonality constraint is doing the work, and every model clears a scrambled-correspondence control and ten further controls. On the two benchmarks of the closest inference-time method, latent-space intervention, run with its own data and metric code, RoSh's gains are five to thirteen times larger.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 25 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

20 of 25 references.

References · 20

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  3. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  4. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  5. 2017Word Translation Without Parallel Datacited by 487
  6. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
  7. 2023Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Modelscited by 118
  8. 2017Offline bilingual word vectors, orthogonal transformations and the inverted softmaxcited by 96
  9. 2024Truth is Universal: Robust Detection of Lies in LLMscited by 79
  10. 2024Bridging the Language Gaps in Large Language Models with Inference-Time Cross-Lingual Interventioncited by 24
  11. 2025Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipelinecited by 21
  12. 2025MuBench: Assessment of Multilingual Capabilities of Large Language Models Across 61 Languagescited by 15
8 more
  1. 2025Improving Multilingual Language Models by Aligning Representations through Steeringcited by 11
  2. 2025Aligning LLMs for Multilingual Consistency in Enterprise Applicationscited by 11
  3. 2025On the Entity-Level Alignment in Crosslingual Consistencycited by 4
  4. 2025Rethinking Cross-lingual Gaps from a Statistical Viewpointcited by 4
  5. 2026BLUFF: Benchmarking the Detection of False and Synthetic Content across 58 Low-Resource Languagescited by 2
  6. 2026Latent-Space Intervention for Cross-Lingual Factual Consistency: Consistency Improvements without Accuracy Dropscited by 2
  7. 2024A Comparative Study of Translation Bias and Accuracy in Multilingual Large Language Models for Cross-Language Claim Verificationcited by 1
  8. 2026DIA-HARM: Dialectal Disparities in Harmful Content Detection Across 50 English Dialectscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning-Guided Claim Normalization for Noisy Multilingual Social Media Posts
  2. 2025Understanding Inequality of LLM Fact-Checking over Geographic Regions with Agent and Retrieval models
  3. 2024FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation
  4. 2023Fine-tuning Language Models for Factuality
  5. 2025Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Models
  6. 2025Are the Hidden States Hiding Something? Testing the Limits of Factuality-Encoding Capabilities in LLMs
  7. 2025CANDY: Benchmarking LLMs' Limitations and Assistive Potential in Chinese Misinformation Fact-Checking
  8. 2025RealFactBench: A Benchmark for Evaluating Large Language Models in Real-World Fact-Checking