mlsanityData to 7 Oct 2026

A Comprehensive Evaluation of Multilingual Chain-of-Thought Reasoning: Performance, Consistency, and Faithfulness Across Languages

arXiv 2510.09555 ↗First posted 10 Oct 2025cs.CLcited by 6

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large reasoning models (LRMs) increasingly rely on step-by-step Chain-of-Thought (CoT) reasoning to improve task performance, particularly in high-resource languages such as English. While recent work has examined final-answer accuracy in multilingual settings, the thinking traces themselves, i.e., the intermediate steps that lead to the final answer, remain underexplored. In this paper, we present the first comprehensive study of multilingual CoT reasoning, evaluating three key dimensions: performance, consistency, and faithfulness. We begin by measuring language compliance, answer accuracy, and answer consistency when LRMs are explicitly instructed or prompt-hacked to think in a target language, revealing strong language preferences and divergent performance across languages. Next, we assess crosslingual consistency of thinking traces by interchanging them between languages. We find that the quality and effectiveness of thinking traces vary substantially depending on the prompt language. Finally, we adapt perturbation-based techniques -- i.e., truncation and error injection -- to probe the faithfulness of thinking traces across languages, showing that models rely on traces to varying degrees. We release our code and data to support future research.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 17 of its 18 references that are in the map. First posted on arXiv 10 Oct 2025.

Builds on

17 of 18 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Language Mixing in Reasoning Language Models: Patterns, Impact, and Internal Causes20 May
  2. 2025Crosslingual Reasoning through Test-Time Scaling8 May
  3. 2025A Survey of Multilingual Reasoning in Language Models13 Feb
  4. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting11 May

Other references · 13

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  5. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  6. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  7. 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languagescited by 96
  8. 2025Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Modelscited by 43
  9. 2024Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thoughtcited by 37
  10. 2025Measuring the Faithfulness of Thinking Drafts in Large Reasoning Modelscited by 22
  11. 2025MAKIEval: A Multilingual Automatic WiKidata-based Framework for Cultural Awareness Evaluation for LLMscited by 13
  12. 2024Systematically Analyzing Prompt Injection Vulnerabilities in Diverse LLM Architecturescited by 12
1 more
  1. 2025Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probingcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices?cited by 10
  2. 2026Reasoning-Trace Collapse: Evaluating the Loss of Explicit Reasoning During Fine-Tuningcited by 4
  3. 2026When Does Language Matter? Multilingual Instructions Reveal Step-wise Language Sensitivity in Vision-Language-Action Modelscited by 3
  4. 2026The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languagescited by 1
  5. 2026Soft Token Alignment for Cross-Lingual Reasoningcited by 0
  6. 2026Does CoT-Pass@k Really Check the CoT? A Multilingual Mathematical Auditcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Rethinking the Multilingual Reasoning Gap with Layer Swap
  2. 2025Long Chain-of-Thought Reasoning Across Languages
  3. 2025AdaMCoT: Rethinking Cross-Lingual Factual Reasoning through Adaptive Multilingual Chain-of-Thought
  4. 2025Towards Better Understanding of Program-of-Thought Reasoning in Cross-Lingual and Multilingual Environments
  5. 2023Measuring Faithfulness in Chain-of-Thought Reasoning
  6. 2025Cross-Lingual Consistency: A Novel Inference Framework for Advancing Reasoning in Large Language Models
  7. 2025Language Matters: How Do Multilingual Input and Reasoning Paths Affect Large Reasoning Models?
  8. 2025Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought