mlsanityData to 14 Oct 2026

Do Multilingual LLMs Think In English?

arXiv 2502.15603 ↗First posted 21 Feb 2025cs.CL cs.AI cs.LGcited by 78

On the map: LLM reasoning → Reasoning benchmarks and evaluation

also in 1 more
Abstract (arXiv)

Large language models (LLMs) have multilingual capabilities and can solve tasks across various languages. However, we show that current LLMs make key decisions in a representation space closest to English, regardless of their input and output languages. Exploring the internal representations with a logit lens for sentences in French, German, Dutch, and Mandarin, we show that the LLM first emits representations close to English for semantically-loaded words before translating them into the target language. We further show that activation steering in these LLMs is more effective when the steering vectors are computed in English rather than in the language of the inputs and outputs. This suggests that multilingual LLMs perform key reasoning steps in a representation that is heavily shaped by English in a way that is not transparent to system users.

Key facts. Cited by 78 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 34 of its 49 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

34 of 49 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Beyond English-Centric LLMs: What Language Do Multilingual Language Models Think in?20 Aug
  2. 2024xCoT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning13 Jan
  3. 2023Are Multilingual LLMs Culturally-Diverse Reasoners? An Investigation into Multicultural Proverbs and Sayings15 Sep
  4. 2023Do Multilingual Language Models Think Better in English?2 Aug
  5. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting11 May
  6. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity8 Feb

Other references · 28

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  3. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  4. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  5. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 530
  6. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  7. 2020How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Modelscited by 318
  8. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
  9. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 243
  10. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 240
  11. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  12. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 225
16 more
  1. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 214
  2. 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 156
  3. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  4. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 143
  5. 2020When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Modelscited by 131
  6. 2023Future Lens: Anticipating Subsequent Tokens from a Single Hidden Statecited by 121
  7. 2024Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilitiescited by 111
  8. 2024Is Cosine-Similarity of Embeddings Really About Similarity?cited by 89
  9. 2024Understanding and Mitigating Language Confusion in LLMscited by 75
  10. 2023Extrapolating Large Language Models to Non-English by Aligning Languagescited by 69
  11. 2024The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalitiescited by 69
  12. 2025Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languagescited by 59
  13. 2024Who's asking? User personas and the mechanics of latent misalignmentcited by 40
  14. 2024Benchmarking Linguistic Diversity of Large Language Modelscited by 38
  15. 2024LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMscited by 33
  16. 2024Multi-FAct: Assessing Factuality of Multilingual LLMs using FActScorecited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 38; 2026: 40.

78 papers.

Sort:

By area:

  1. 2026Verbalizable Representations Form a Global Workspace in Language Modelscited by 75
  2. 2025Refusal Direction is Universal Across Safety-Aligned Languagescited by 38
  3. 2025The Homogenizing Effect of Large Language Models on Human Expression and Thoughtcited by 35
  4. 2025The Dual-Route Model of Inductioncited by 24
  5. 2025Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoningcited by 24
  6. 2025R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learningcited by 21
  7. 2025Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipelinecited by 21
  8. 2025The Emergence of Abstract Thought in Large Language Models Beyond Any Languagecited by 21
  9. 2025Localized Cultural Knowledge is Conserved and Controllable in Large Language Modelscited by 20
  10. 2025Language Matters: How Do Multilingual Input and Reasoning Paths Affect Large Reasoning Models?cited by 20
  11. 2025Language Arithmetics: Towards Systematic Language Neuron Identification and Manipulationcited by 17
  12. 2025Understanding Language Prior of LVLMs by Contrasting Chain-of-Embeddingcited by 14
38 more
  1. 2025Beyond Transcription: Mechanistic Interpretability in ASRcited by 13
  2. 2025Cross-lingual Collapse: How Language-Centric Foundation Models Shape Reasoning in Large Language Modelscited by 11
  3. 2025The Translation Barrier Hypothesis: Multilingual Generation with Large Language Models Suffers from Implicit Translation Failurecited by 10
  4. 2026Understanding Multilingualism in Mixture-of-Experts LLMs: Routing Mechanism, Expert Specialization, and Layerwise Steeringcited by 10
  5. 2025CCL-XCoT: An Efficient Cross-Lingual Knowledge Transfer Method for Mitigating Hallucination Generationcited by 9
  6. 2025Activation Steering for Masked Diffusion Language Modelscited by 9
  7. 2025Tracing Multilingual Representations in LLMs with Cross-Layer Transcoderscited by 8
  8. 2025Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAGcited by 7
  9. 2025TigerCoder: A Novel Suite of LLMs for Code Generation in Banglacited by 6
  10. 2025Beyond Early-Token Bias: Model-Specific and Language-Specific Position Effects in Multilingual LLMscited by 5
  11. 2025What Language(s) Does Aya-23 Think In? How Multilinguality Affects Internal Language Representationscited by 5
  12. 2026When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Trainingcited by 5
  13. 2025Language Specific Knowledge: Do Models Know Better in X than in English?cited by 4
  14. 2025Under the Shadow of Babel: How Language Shapes Reasoning in LLMscited by 4
  15. 2025Investigating Bias: A Multilingual Pipeline for Generating, Solving, and Evaluating Math Problems with LLMscited by 4
  16. 2025On the Entity-Level Alignment in Crosslingual Consistencycited by 4
  17. 2025Text2Cypher Across Languages: Evaluating and Finetuning LLMscited by 3
  18. 2025Humans overrely on overconfident language models, across languagescited by 3
  19. 2025Beyond the Rosetta Stone: Unification Forces in Generalization Dynamicscited by 3
  20. 2025Controlling the Risk of Corrupted Contexts for Language Models via Early-Exitingcited by 3
  21. 2025Gradual Code-Switching as Inference-Time Cross-Lingual Representational Alignment for LLMscited by 3
  22. 2025Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Modelscited by 3
  23. 2026OLA: Output Language Alignment in Code-Switched LLM Interactionscited by 3
  24. 2026LinguaMap: Which Layers of LLMs Speak Your Language and How to Tune Them?cited by 3
  25. 2026Cross-Lingual LLM-Judge Transfer via Evaluation Decompositioncited by 3
  26. 2026Multilingual Language Models Encode Script Over Linguistic Structurecited by 3
  27. 2025Diagnosing Vision Language Models' Perception by Leveraging Human Methods for Color Vision Deficienciescited by 2
  28. 2025The Riddle of Reflection: Evaluating Reasoning and Self-Awareness in Multilingual LLMs using Indian Riddlescited by 2
  29. 2026Med-CoReasoner: Reducing Language Disparities in Medical Reasoning via Language-Informed Co-Reasoningcited by 2
  30. 2026A Shared Geometry of Difficulty in Multilingual Language Modelscited by 2
  31. 2026Typhoon-S: Minimal Open Post-Training for Sovereign Large Language Modelscited by 2
  32. 2026Indic-TunedLens: Interpreting Multilingual Models in Indian Languagescited by 2
  33. 2026Rethinking the Multilingual Reasoning Gap with Layer Swapcited by 2
  34. 2025Do LLMs Need to Think in One Language? Correlation between Latent Language and Task Performancecited by 1
  35. 2025Vocab Diet: Reshaping the Vocabulary of LLMs via Vector Arithmeticcited by 1
  36. 2026Multilingual Large Language Models do not comprehend all natural languages to equal degreescited by 1
  37. 2026Shared Doubt: Zero-Shot Cross-Lingual Confidence Estimation for Language Modelscited by 1
  38. 2026PolyFact: Comparing Consistency-Driven Post-training Methods for Cross-Lingual Factual Recallcited by 1

50 most cited of 78 All 78 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Language-Specific Latent Process Hinders Cross-Lingual Performance
  2. 2025EMCEE: Improving Multilingual Capability of LLMs via Bridging Knowledge and Reasoning with Extracted Synthetic Multilingual Context
  3. 2025Can you map it to English? The Role of Cross-Lingual Alignment in Multilingual Performance of LLMs
  4. 2024Understanding the role of FFNs in driving multilingual behaviour in LLMs
  5. 2025Could Thinking Multilingually Empower LLM Reasoning?
  6. 2025Improving Multilingual Language Models by Aligning Representations through Steering
  7. 2025Assessing Agentic Large Language Models in Multilingual National Bias
  8. 2026The Interlingua Hypothesis: LLMs Translate via a Latent Task-agnostic Feature Space