mlsanityData to 14 Oct 2026

Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models

arXiv 2310.10378 ↗First posted 16 Oct 2023cs.CL cs.AI cs.HCcited by 118

Not on the map. Why?

Abstract (arXiv)

Multilingual large-scale Pretrained Language Models (PLMs) have been shown to store considerable amounts of factual knowledge, but large variations are observed across languages. With the ultimate goal of ensuring that users with different language backgrounds obtain consistent feedback from the same model, we study the cross-lingual consistency (CLC) of factual knowledge in various multilingual PLMs. To this end, we propose a Ranking-based Consistency (RankC) metric to evaluate knowledge consistency across languages independently from accuracy. Using this metric, we conduct an in-depth analysis of the determining factors for CLC, both at model level and at language-pair level. Among other results, we find that increasing model size leads to higher factual probing accuracy in most languages, but does not improve cross-lingual consistency. Finally, we conduct a case study on CLC when new factual associations are inserted in the PLMs via model editing. Results on a small sample of facts inserted in English reveal a clear pattern whereby the new piece of knowledge transfers only to languages with which English has a high RankC score.

Key facts. Cited by 118 papers in the map (data through 14 Oct 2026). Builds on 31 of its 39 references that are in the map. First posted on arXiv 16 Oct 2023.

Builds on

31 of 39 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  6. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  7. 2019Language Models as Knowledge Bases?cited by 1,936
  8. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  9. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  10. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  11. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  12. 2021Editing Factual Knowledge in Language Modelscited by 553
19 more
  1. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  2. 2021Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classificationcited by 183
  3. 2022Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Modelscited by 180
  4. 2021Efficient Large Scale Language Modeling with Mixtures of Expertscited by 175
  5. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  6. 2020X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Modelscited by 117
  7. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
  8. 2022Language Contamination Helps Explain the Cross-lingual Capabilities of English Pretrained Modelscited by 92
  9. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  10. 2019Inducing Relational Knowledge from BERTcited by 75
  11. 2019A Logic-Driven Framework for Consistency of Neural Modelscited by 64
  12. 2018Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processingcited by 63
  13. 2021Knowledge Inheritance for Pre-trained Language Modelscited by 38
  14. 2022GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Modelscited by 30
  15. 2022COPEN: Probing Conceptual Knowledge in Pre-trained Language Modelscited by 21
  16. 2022Factual Consistency of Multilingual Pretrained Language Modelscited by 15
  17. 2023Separating form and meaning: Using self-consistency to quantify task understanding across multiple sensescited by 15
  18. 2021Probing Pre-Trained Language Models for Disease Knowledgecited by 10
  19. 2022Adapters for Enhanced Modeling of Multilingual Knowledge and Textcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 39; 2025: 47; 2026: 32.

118 papers.

Sort:

By area:

  1. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
  2. 2024Investigating Cultural Alignment of Large Language Modelscited by 143
  3. 2024Factual Confidence of LLMs: on Reliability and Robustness of Current Estimatorscited by 61
  4. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  5. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 50
  6. 2025Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Modelscited by 44
  7. 2024MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMscited by 35
  8. 2024Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspectivecited by 29
  9. 2025ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfercited by 29
  10. 2024Multilingual Pretraining and Instruction Tuning Improve Cross-Lingual Knowledge Alignment, But Only Shallowlycited by 27
  11. 2024Code-Switching Curriculum Learning for Multilingual Transfer in LLMscited by 27
  12. 2025MultiLoKo: a multilingual local knowledge benchmark for LLMs spanning 31 languagescited by 26
38 more
  1. 2024Exploring Multilingual Concepts of Human Value in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages?cited by 24
  2. 2025BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Modelscited by 23
  3. 2025Investigating Language Preference of Multilingual RAG Systemscited by 22
  4. 2024Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMscited by 21
  5. 2025Language-Specific Latent Process Hinders Cross-Lingual Performancecited by 21
  6. 2025Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipelinecited by 21
  7. 2024Cross-Lingual Unlearning of Selective Knowledge in Multilingual Language Modelscited by 20
  8. 2024PreAlign: Boosting Cross-Lingual Transfer by Early Establishment of Multilingual Alignmentcited by 20
  9. 2025Localized Cultural Knowledge is Conserved and Controllable in Large Language Modelscited by 20
  10. 2024The SIFo Benchmark: Investigating the Sequential Instruction Following Ability of Large Language Modelscited by 18
  11. 2024Large Language Models Are Cross-Lingual Knowledge-Free Reasonerscited by 16
  12. 2024Cross-Care: Assessing the Healthcare Implications of Pre-training Data on Language Model Biascited by 15
  13. 2025MuBench: Assessment of Multilingual Capabilities of Large Language Models Across 61 Languagescited by 15
  14. 2025Latent learning: episodic memory complements parametric learning by enabling flexible reuse of experiencescited by 15
  15. 2024Teaching LLMs to Abstain across Languages via Multilingual Feedbackcited by 14
  16. 2024Multilingual Retrieval Augmented Generation for Culturally-Sensitive Tasks: A Benchmark for Cross-lingual Robustnesscited by 14
  17. 2025AdaMCoT: Rethinking Cross-Lingual Factual Reasoning through Adaptive Multilingual Chain-of-Thoughtcited by 14
  18. 2024Knowledge Localization: Mission Not Accomplished? Enter Query Localization!cited by 13
  19. 2024BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learningcited by 13
  20. 2024Concept Space Alignment in Multilingual LLMscited by 13
  21. 2025MAKIEval: A Multilingual Automatic WiKidata-based Framework for Cultural Awareness Evaluation for LLMscited by 13
  22. 2024Exploring Multilingual Probing in Large Language Models: A Cross-Language Analysiscited by 12
  23. 2025XCOMPS: A Multilingual Benchmark of Conceptual Minimal Pairscited by 12
  24. 2025Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metricscited by 12
  25. 2024CrossIn: An Efficient Instruction Tuning Approach for Cross-Lingual Knowledge Alignmentcited by 11
  26. 2025CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answeringcited by 10
  27. 2025Mass-Editing Memory with Attention in Transformers: A cross-lingual exploration of knowledgecited by 10
  28. 2025On the Consistency of Multilingual Context Utilization in Retrieval-Augmented Generationcited by 10
  29. 2025Tracing Multilingual Factual Knowledge Acquisition in Pretrainingcited by 10
  30. 2025BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Datacited by 10
  31. 2024MPN: Leveraging Multilingual Patch Neuron for Cross-lingual Model Editingcited by 9
  32. 2024ERBench: An Entity-Relationship based Automatically Verifiable Hallucination Benchmark for Large Language Modelscited by 9
  33. 2024Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Modelscited by 9
  34. 2025From Neurons to Semantics: Evaluating Cross-Linguistic Alignment Capabilities of Large Language Models via Neurons Alignmentcited by 9
  35. 2025Do You Know About My Nation? Investigating Multilingual Language Models' Cultural Literacy Through Factual Knowledgecited by 8
  36. 2024From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistencycited by 7
  37. 2024Large Language Models are Limited in Out-of-Context Knowledge Reasoningcited by 6
  38. 2025Geopolitical biases in LLMs: what are the "good" and the "bad" countries according to contemporary language modelscited by 6

50 most cited of 118 All 118 (JSON)