# Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineering

arXiv: https://arxiv.org/abs/2511.23231 · HTML: https://mlsanity.com/paper/2511.23231/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 25 of its 35 references that are in the map.
- First posted on arXiv 28 Nov 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)
- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)
- Mechanistic interpretability / [Steering and interpreting LLM reasoning](https://mlsanity.com/mechinterp/steering-and-interpreting-llm-reasoning/)

## Builds on (most cited first, 25 of 25)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [Language Models are Multilingual Chain-of-Thought Reasoners](https://mlsanity.com/paper/2210.03057/) (2022)
- [Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations](https://mlsanity.com/paper/2310.20246/) (2023)
- [PolyMath: Evaluating Mathematical Reasoning in Multilingual Contexts](https://mlsanity.com/paper/2504.18428/) (2025)
- [Bring Reason to Vision: Understanding Perception and Reasoning through Model Merging](https://mlsanity.com/paper/2505.05464/) (2025)
- [Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering](https://mlsanity.com/paper/2503.11314/) (2025)
- [Improving In-context Learning of Multilingual Generative Language Models with Cross-lingual Alignment](https://mlsanity.com/paper/2311.08089/) (2023)
- [mCoT: Multilingual Instruction Tuning for Reasoning Consistency in Language Models](https://mlsanity.com/paper/2406.02301/) (2024)
- [Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models](https://mlsanity.com/paper/2403.10258/) (2024)
- [Causal Tracing of Object Representations in Large Vision Language Models: Mechanistic Interpretability and Hallucination Mitigation](https://mlsanity.com/paper/2511.05923/) (2025)
- [Long Chain-of-Thought Reasoning Across Languages](https://mlsanity.com/paper/2508.14828/) (2025)
- [SLAM: Towards Efficient Multilingual Reasoning via Selective Language Alignment](https://mlsanity.com/paper/2501.03681/) (2025)
- [When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners](https://mlsanity.com/paper/2505.15257/) (2025)
- [M4U: Evaluating Multilingual Understanding and Reasoning for Large Multimodal Models](https://mlsanity.com/paper/2405.15638/) (2024)
- [CLAIM: Mitigating Multilingual Object Hallucination in Large Vision-Language Models with Cross-Lingual Attention Intervention](https://mlsanity.com/paper/2506.11073/) (2025)
- [mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humans](https://mlsanity.com/paper/2406.04215/) (2024)
- [Do Moral Judgment and Reasoning Capability of LLMs Change with Language? A Study using the Multilingual Defining Issues Test](https://mlsanity.com/paper/2402.02135/) (2024)
- [Why Not Transform Chat Large Language Models to Non-English?](https://mlsanity.com/paper/2405.13923/) (2024)
- [CAI: Caption-Sensitive Attention Intervention for Mitigating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2506.23590/) (2025)
- [Towards Robust Knowledge Representations in Multilingual LLMs for Equivalence and Inheritance based Consistent Reasoning](https://mlsanity.com/paper/2410.14235/) (2024)

## Built on by (most cited first, 8 of 8)

- [MPR-GUI: Benchmarking and Enhancing Multilingual Perception and Reasoning in GUI Agents](https://mlsanity.com/paper/2512.00756/) (2025)
- [I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editing](https://mlsanity.com/paper/2601.03741/) (2026)
- [When Does Language Matter? Multilingual Instructions Reveal Step-wise Language Sensitivity in Vision-Language-Action Models](https://mlsanity.com/paper/2606.11906/) (2026)
- [G2TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models](https://mlsanity.com/paper/2605.12309/) (2026)
- [Efficient Multilingual Reasoning Transfer via Progressive Code-Switching](https://mlsanity.com/paper/2607.00485/) (2026)
- [Analyzing and Mitigating Cross-Lingual Degradation in Multilingual Medical VQA](https://mlsanity.com/paper/2608.22363/) (2026)
- [Cross-Lingual Representation Alignment by Token-Level Optimal Transport in a Language-Agnostic Space](https://mlsanity.com/paper/2609.06381/) (2026)
- [Language-Conditioned Token and Reasoning Efficiency in Large Language Models: A Paired Cross-Lingual Study Protocol](https://mlsanity.com/paper/2610.04295/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) and Large Vision-Language Models (LVLMs) demonstrate strong reasoning capabilities, yet their performance in English significantly outperforms that in low-resource languages, raising fairness concerns in multilingual applications. Existing approaches either rely on costly multilingual training or employ prompting with external translation tools, both of which are resource-intensive and sensitive to translation quality. To address these limitations, we propose a training-free inference-time method to enhance Multilingual Reasoning capabilities via Representation Engineering (MRRE) without using any additional training data or tools. MRRE sequentially injects two precomputed vectors at specific layers during inference processing: cross-lingual reasoning enhancement vectors, which steer non-English reasoning representations toward English space to unlock multilingual reasoning, and target-language output anchoring vectors, which restore the distribution of the target language to preserve input-output language consistency. Comprehensive experiments across six advanced LLMs and LVLMs on four reasoning benchmarks demonstrate that MRRE consistently enhances non-English reasoning by an average gain of 5.48% and up to 7.54% in low-resource languages (Thai and Swahili), while improving input-output language consistency by 3.78%.