mlsanityData to 14 Oct 2026

Multilingual Test-Time Scaling via Initial Thought Transfer

arXiv 2505.15508 ↗First posted 21 May 2025cs.CLcited by 5

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Test-time scaling has emerged as a widely adopted inference-time strategy for boosting reasoning performance. However, its effectiveness has been studied almost exclusively in English, leaving its behavior in other languages largely unexplored. We present the first systematic study of test-time scaling in multilingual settings, evaluating DeepSeek-R1-Distill-LLama-8B and DeepSeek-R1-Distill-Qwen-7B across both high- and low-resource Latin-script languages. Our findings reveal that the relative gains from test-time scaling vary significantly across languages. Additionally, models frequently switch to English mid-reasoning, even when operating under strictly monolingual prompts. We further show that low-resource languages not only produce initial reasoning thoughts that differ significantly from English but also have lower internal consistency across generations in their early reasoning. Building on our findings, we introduce MITT (Multilingual Initial Thought Transfer), an unsupervised and lightweight reasoning prefix-tuning approach that transfers high-resource reasoning prefixes to enhance test-time scaling across all languages, addressing inconsistencies in multilingual reasoning performance. MITT significantly boosts DeepSeek-R1-Distill-Qwen-7B's reasoning performance, especially for underrepresented languages.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 32 of its 39 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

32 of 39 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025From System 1 to System 2: A Survey of Reasoning Large Language Models24 Feb
  2. 2025A Survey of Multilingual Reasoning in Language Models13 Feb
  3. 2024Multi-Step Reasoning with Large Language Models, a Survey16 Jul

Other references · 29

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2024GPT-4o System Cardcited by 4,588
  7. 2023Qwen Technical Reportcited by 3,344
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  9. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  10. 2025s1: Simple test-time scalingcited by 1,341
  11. 2024OpenAI o1 System Cardcited by 1,244
  12. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
17 more
  1. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 593
  2. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 530
  3. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  4. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 262
  5. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 240
  6. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
  7. 2022Will we run out of data? Limits of LLM scaling based on human-generated datacited by 195
  8. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scalingcited by 136
  9. 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!cited by 132
  10. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoningcited by 119
  11. 2024The rising costs of training frontier AI modelscited by 95
  12. 2025Atom of Thoughts for Markov LLM Test-Time Scalingcited by 54
  13. 2025Large Language Models Pass the Turing Testcited by 48
  14. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  15. 2020A Review of Winograd Schema Challenge Datasets and Approachescited by 27
  16. 2022Crosslingual Generalization through Multitask Finetuningcited by 26
  17. 2025From Drafts to Answers: Unlocking LLM Potential via Aggregation Fine-Tuningcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Why Do Multilingual Reasoning Gaps Emerge in Reasoning Language Models?cited by 12
  2. 2026Language of Thought Shapes Output Diversity in Large Language Modelscited by 4
  3. 2025EfficientXLang: Towards Improving Token Efficiency Through Cross-Lingual Reasoningcited by 3
  4. 2026VEPO: Variable Entropy Policy Optimization for Low-Resource Language Foundation Modelscited by 0
  5. 2026When Does Reasoning Help in Machine Translation? A Hierarchical Analysis of LRM Reasoning Tracescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Crosslingual Reasoning through Test-Time Scaling
  2. 2026Rethinking the Multilingual Reasoning Gap with Layer Swap
  3. 2025The Art of Scaling Test-Time Compute for Large Language Models
  4. 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinking
  5. 2025A Comprehensive Evaluation of Multilingual Chain-of-Thought Reasoning: Performance, Consistency, and Faithfulness Across Languages
  6. 2025AdaMCoT: Rethinking Cross-Lingual Factual Reasoning through Adaptive Multilingual Chain-of-Thought
  7. 2025Long Chain-of-Thought Reasoning Across Languages
  8. 2025Rethinking Inference-Time Scaling: Efficiency Limits and Linguistic Signals