mlsanityData to 7 Oct 2026

Can you map it to English? The Role of Cross-Lingual Alignment in Multilingual Performance of LLMs

arXiv 2504.09378 ↗First posted 13 Apr 2025cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) can answer prompts in many languages, despite being trained predominantly on English; yet, the mechanisms driving this generalization remain poorly understood. This work asks: How does an LLM's ability to align representations of non-English inputs to English impact its performance on natural language understanding (NLU) tasks? We study the role of representation alignment in instance-level task decisions, complementing prior analyses conducted both at the language level and task-independently. We introduce the Discriminative Alignment Index (DALI) to quantify instance-level alignment across 24 languages other than English and three distinct NLU tasks. Results show that incorrect NLU predictions are strongly associated with lower representation alignment with English in the model's middle layers. Through activation patching, we show that incorrect predictions in languages other than English can be fixed by patching their parallel English activations in the middle layers, thereby demonstrating the causal role of representation (mis)alignment in cross-lingual correctness.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 23 of its 38 references that are in the map. First posted on arXiv 13 Apr 2025.

Builds on

23 of 38 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  3. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  4. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  5. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  6. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  7. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  8. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
  9. 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 262
  10. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 186
  11. 2020Translation Artifacts in Cross-lingual Transfer Learningcited by 95
  12. 2023Do Multilingual Language Models Think Better in English?cited by 87
11 more
  1. 2025Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Modelscited by 43
  2. 2024Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformerscited by 42
  3. 2024Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languagescited by 38
  4. 2024Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questionscited by 35
  5. 2024Understanding Cross-Lingual Alignment -- A Surveycited by 34
  6. 2023Improving In-context Learning of Multilingual Generative Language Models with Cross-lingual Alignmentcited by 33
  7. 2025Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMscited by 26
  8. 2024MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignmentcited by 25
  9. 2024Bridging the Language Gaps in Large Language Models with Inference-Time Cross-Lingual Interventioncited by 24
  10. 2025Steering into New Embedding Spaces: Analyzing Cross-Lingual Alignment Induced by Model Interventions in Multilingual Language Modelscited by 10
  11. 2025High-Dimensional Interlingual Representations of Large Language Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 6.

11 papers.

  1. 2025Multilingual Routing in Mixture-of-Expertscited by 25
  2. 2025One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizerscited by 11
  3. 2025Rethinking Cross-lingual Alignment: Balancing Transfer and Cultural Erasure in Multilingual LLMscited by 7
  4. 2025Less Data Less Tokens: Multilingual Unification Learning for Efficient Test-Time Reasoning in LLMscited by 3
  5. 2026Can Embedding Similarity Predict Cross-Lingual Transfer? A Systematic Study on African Languagescited by 3
  6. 2026Learning to Route Languages for Multilingual Policy Optimizationcited by 1
  7. 2025Discriminating Form and Meaning in Multilingual Models with Minimal-Pair ABX Taskscited by 0
  8. 2026A Family of LLMs Liberated from Static Vocabulariescited by 0
  9. 2026Bridging the English-Arabic Medical Knowledge Gap: Targeted Low-Rank Adaptation via Causal Layer Selectioncited by 0
  10. 2026A Systematic Comparison of Multilingual Interpretability Methods Reveals Anisotropy-Driven Failurescited by 0
  11. 2026Cross-Lingual Alignment for Decoder-Only Models using MoE Routerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025How Language Directions Align with Token Geometry in Multilingual LLMs
  2. 2026LLM Alignment--Utility Asymmetry under Semantic-Preserving Transformations
  3. 2025Do Multilingual LLMs Think In English?
  4. 2026Cross-Lingual Alignment Without Joint Training: Do Monolingual Language Models Converge on Universal Representations?
  5. 2025From Neurons to Semantics: Evaluating Cross-Linguistic Alignment Capabilities of Large Language Models via Neurons Alignment
  6. 2024Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Models
  7. 2025The Hidden Space of Safety: Understanding Preference-Tuned LLMs in Multilingual context
  8. 2024Probing the Emergence of Cross-lingual Alignment during LLM Training