# How Do Multilingual Language Models Remember Facts?

arXiv: https://arxiv.org/abs/2410.14387 · HTML: https://mlsanity.com/paper/2410.14387/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Oct 2024.

## Built on by (most cited first, 20 of 20)

- [Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers](https://mlsanity.com/paper/2411.08745/) (2024)
- [Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipeline](https://mlsanity.com/paper/2505.20546/) (2025)
- [The Emergence of Abstract Thought in Large Language Models Beyond Any Language](https://mlsanity.com/paper/2506.09890/) (2025)
- [LatentLens: Revealing Highly Interpretable Visual Tokens in LLMs](https://mlsanity.com/paper/2602.00462/) (2026)
- [Language Models' Factuality Depends on the Language of Inquiry](https://mlsanity.com/paper/2502.17955/) (2025)
- [Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing](https://mlsanity.com/paper/2507.10587/) (2025)
- [Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models](https://mlsanity.com/paper/2505.16538/) (2025)
- [Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networks](https://mlsanity.com/paper/2511.19265/) (2025)
- [On the Entity-Level Alignment in Crosslingual Consistency](https://mlsanity.com/paper/2510.10280/) (2025)
- [Rethinking Cross-lingual Gaps from a Statistical Viewpoint](https://mlsanity.com/paper/2510.15551/) (2025)
- [Iterative Multilingual Spectral Attribute Erasure](https://mlsanity.com/paper/2506.11244/) (2025)
- [How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation](https://mlsanity.com/paper/2507.20758/) (2025)
- [Two Pathways to Truthfulness: On the Intrinsic Encoding of LLM Hallucinations](https://mlsanity.com/paper/2601.07422/) (2026)
- [Polyglots or Multitudes? Multilingual LLM Answers to Value-laden Multiple-Choice Questions](https://mlsanity.com/paper/2602.05932/) (2026)
- [Divergent large language model predictions from convergent representations in ambiguous word pairs](https://mlsanity.com/paper/2608.01816/) (2026)
- [Who Says What: Symbolic Trimodal Binding Mechanisms in Audio-Visual LLMs](https://mlsanity.com/paper/2609.31193/) (2026)
- [Medical Interpretability and Knowledge Maps of Large Language Models](https://mlsanity.com/paper/2510.11390/) (2025)
- [FactNet: A Billion-Scale Knowledge Graph for Multilingual Factual Grounding](https://mlsanity.com/paper/2602.03417/) (2026)
- [Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation](https://mlsanity.com/paper/2604.19678/) (2026)
- [Same Values, Different Languages? From Multilingual Probing to Steering LLMs Toward Chinese Social Values](https://mlsanity.com/paper/2609.08515/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) store and retrieve vast amounts of factual knowledge acquired during pre-training. Prior research has localized and identified mechanisms behind knowledge recall; however, it has only focused on English monolingual models. The question of how these mechanisms generalize to non-English languages and multilingual LLMs remains unexplored. In this paper, we address this gap by conducting a comprehensive analysis of three multilingual LLMs. First, we show that previously identified recall mechanisms in English largely apply to multilingual contexts, with nuances based on language and architecture. Next, through patching intermediate representations, we localize the role of language during recall, finding that subject enrichment is language-independent, while object extraction is language-dependent. Additionally, we discover that the last token representation acts as a Function Vector (FV), encoding both the language of the query and the content to be extracted from the subject. Furthermore, in decoder-only LLMs, FVs compose these two pieces of information in two separate stages. These insights reveal unique mechanisms in multilingual LLMs for recalling information, highlighting the need for new methodologies -- such as knowledge evaluation, fact editing, and knowledge acquisition -- that are specifically tailored for multilingual LLMs.