mlsanityData to 14 Oct 2026

Language Representation Projection: Can We Transfer Factual Knowledge across Languages in Multilingual Language Models?

arXiv 2311.03788 ↗First posted 7 Nov 2023cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

Multilingual pretrained language models serve as repositories of multilingual factual knowledge. Nevertheless, a substantial performance gap of factual knowledge probing exists between high-resource languages and low-resource languages, suggesting limited implicit factual knowledge transfer across languages in multilingual pretrained language models. This paper investigates the feasibility of explicitly transferring relatively rich factual knowledge from English to non-English languages. To accomplish this, we propose two parameter-free Language Representation Projection modules (LRP2). The first module converts non-English representations into English-like equivalents, while the second module reverts English-like representations back into representations of the corresponding non-English language. Experimental results on the mLAMA dataset demonstrate that LRP2 significantly improves factual knowledge retrieval accuracy and facilitates knowledge transferability across diverse non-English languages. We further investigate the working mechanism of LRP2 from the perspectives of representation space and cross-lingual knowledge neuron.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 22 of its 30 references that are in the map. First posted on arXiv 7 Nov 2023.

Builds on

22 of 30 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  4. 2019Language Models as Knowledge Bases?cited by 1,936
  5. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  6. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  7. 2019How Can We Know What Language Models Know?cited by 959
  8. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  9. 2021Editing Factual Knowledge in Language Modelscited by 553
  10. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  11. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 340
  12. 2021Few-shot Learning with Multilingual Language Modelscited by 278
10 more
  1. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  2. 2020InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Trainingcited by 237
  3. 2020Editable Neural Networkscited by 155
  4. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  5. 2020X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Modelscited by 117
  6. 2020On the Language Neutrality of Pre-trained Multilingual Representationscited by 85
  7. 2020Inducing Language-Agnostic Multilingual Representationscited by 48
  8. 2021Consistency Regularization for Cross-Lingual Fine-Tuningcited by 41
  9. 2022Enhancing Cross-lingual Transfer by Manifold Mixupcited by 30
  10. 2022Language Anisotropic Cross-Lingual Model Editingcited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 7; 2026: 3.

19 papers.

Sort:

By area:

  1. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 126
  2. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  3. 2024Exploring Multilingual Concepts of Human Value in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages?cited by 24
  4. 2024Multilingual Large Language Models: A Systematic Surveycited by 18
  5. 2024Teaching LLMs to Abstain across Languages via Multilingual Feedbackcited by 14
  6. 2024Concept Space Alignment in Multilingual LLMscited by 13
  7. 2025Enhancing Non-English Capabilities of English-Centric Large Language Models through Deep Supervision Fine-Tuningcited by 11
  8. 2024FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Datacited by 10
  9. 2024Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Modelscited by 9
  10. 2025Analyzing LLMs' Knowledge Boundary Cognition Across Languages Through the Lens of Internal Representationscited by 6
  11. 2025Decoding Knowledge Attribution in Mixture-of-Experts: A Framework of Basic-Refinement Collaboration and Efficiency Analysiscited by 5
  12. 2025Language Fusion for Parameter-Efficient Cross-lingual Transfercited by 3
7 more
  1. 2024Pruning Multilingual Large Language Models for Multilingual Inferencecited by 2
  2. 2026Latent-Space Intervention for Cross-Lingual Factual Consistency: Consistency Improvements without Accuracy Dropscited by 2
  3. 2025Interpreting Social Bias in LVLMs via Information Flow Analysis and Multi-Round Dialogue Evaluationcited by 1
  4. 2025Enhancing Cross-Lingual Transfer through Reversible Transliteration: A Huffman-Based Approach for Low-Resource Languagescited by 1
  5. 2026Code-Switching Reveals Language Anchoring in Multilingual LLMscited by 1
  6. 2025Assessing Agentic Large Language Models in Multilingual National Biascited by 0
  7. 2026Probing Factual Knowledge Transfer with Training Data Interventionscited by 0