mlsanityData to 14 Oct 2026

Question Translation Training for Better Multilingual Reasoning

arXiv 2401.07817 ↗First posted 15 Jan 2024cs.CLcited by 42

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large language models show compelling performance on reasoning tasks but they tend to perform much worse in languages other than English. This is unsurprising given that their training data largely consists of English text and instructions. A typical solution is to translate instruction data into all languages of interest, and then train on the resulting multilingual data, which is called translate-training. This approach not only incurs high cost, but also results in poorly translated data due to the non-standard formatting of mathematical chain-of-thought. In this paper, we explore the benefits of question alignment, where we train the model to translate reasoning questions into English by finetuning on X-English parallel question data. In this way we perform targeted, in-domain language alignment which makes best use of English instruction data to unlock the LLMs' multilingual reasoning abilities. Experimental results on LLaMA2-13B show that question alignment leads to consistent improvements over the translate-training approach: an average improvement of 11.3% and 16.1% accuracy across ten languages on the MGSM and MSVAMP multilingual reasoning benchmarks. The project will be available at: https://github.com/NJUNLP/QAlign.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 19 of its 24 references that are in the map. First posted on arXiv 15 Jan 2024.

Builds on

19 of 24 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting11 May
  2. 2023LLaMA: Open and Efficient Foundation Language Models27 Feb

Other references · 17

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  5. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  6. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  7. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  8. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  9. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 530
  10. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487
  11. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 408
  12. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 207
5 more
  1. 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languagescited by 97
  2. 2022Language Contamination Helps Explain the Cross-lingual Capabilities of English Pretrained Modelscited by 92
  3. 2023Extrapolating Large Language Models to Non-English by Aligning Languagescited by 69
  4. 2023Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observationscited by 59
  5. 2023Revisiting Machine Translation for Cross-lingual Classificationcited by 41

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 24; 2026: 7.

42 papers.

Sort:

By area:

  1. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
  2. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  3. 2024LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languagescited by 48
  4. 2025Crosslingual Reasoning through Test-Time Scalingcited by 46
  5. 2025Understand, Solve and Translate: Bridging the Multilingual Mathematical Reasoning Gapcited by 43
  6. 2025Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoningcited by 37
  7. 2025A Survey of Multilingual Reasoning in Language Modelscited by 33
  8. 2024MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignmentcited by 25
  9. 2024MindMerger: Efficient Boosting LLM Reasoning in non-English Languagescited by 21
  10. 2025Language-Specific Latent Process Hinders Cross-Lingual Performancecited by 21
  11. 2024PreAlign: Boosting Cross-Lingual Transfer by Early Establishment of Multilingual Alignmentcited by 20
  12. 2024Multilingual Large Language Models: A Systematic Surveycited by 18
30 more
  1. 2025SLAM: Towards Efficient Multilingual Reasoning via Selective Language Alignmentcited by 18
  2. 2024Large Language Models Are Cross-Lingual Knowledge-Free Reasonerscited by 16
  3. 2025Reasoning Beyond Limits: Advances and Open Problems for LLMscited by 16
  4. 2025AdaMCoT: Rethinking Cross-Lingual Factual Reasoning through Adaptive Multilingual Chain-of-Thoughtcited by 14
  5. 2025Why Do Multilingual Reasoning Gaps Emerge in Reasoning Language Models?cited by 12
  6. 2024Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learnerscited by 11
  7. 2025Improving Multilingual Language Models by Aligning Representations through Steeringcited by 11
  8. 2025Cross-lingual Collapse: How Language-Centric Foundation Models Shape Reasoning in Large Language Modelscited by 11
  9. 2025Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thoughtcited by 10
  10. 2025When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMscited by 9
  11. 2025From Neurons to Semantics: Evaluating Cross-Linguistic Alignment Capabilities of Large Language Models via Neurons Alignmentcited by 9
  12. 2025How Does Alignment Enhance LLMs' Multilingual Capabilities? A Language Neurons Perspectivecited by 8
  13. 2024The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insightscited by 7
  14. 2024Lens: Rethinking Multilingual Enhancement for Large Language Modelscited by 6
  15. 2025Towards Better Understanding of Program-of-Thought Reasoning in Cross-Lingual and Multilingual Environmentscited by 6
  16. 2025AlignX: Advancing Multilingual Large Language Models with Multilingual Representation Alignmentcited by 6
  17. 2025Benchmarking and Boosting Multilingual Capabilities of LVLMs via OCR-Centric Reinforcement Learningcited by 4
  18. 2025Understanding LLMs' Cross-Lingual Context Retrieval: How Good It Is And Where It Comes Fromcited by 4
  19. 2025Language-Specific Layer Matters: Efficient Multilingual Enhancement for Large Vision-Language Modelscited by 3
  20. 2025High-Dimensional Interlingual Representations of Large Language Modelscited by 2
  21. 2025MERLIN: Multi-Stage Curriculum Alignment for Multilingual Encoder-LLM Integration in Cross-Lingual Reasoningcited by 2
  22. 2025Are the LLMs Capable of Maintaining at Least the Language Genus?cited by 1
  23. 2026No One Fits All: From Fixed Prompting to Learned Routing in Multilingual LLMscited by 1
  24. 2025Utilizing Multilingual Encoders to Improve Large Language Models for Low-Resource Languagescited by 0
  25. 2026Enhancing Multilingual Reasoning via Steerable Model Mergingcited by 0
  26. 2026RP-OPSD: Reasoning-Pivot-Guided On-Policy Self-Distillation for Multilingual Reasoning Transfercited by 0
  27. 2026Choosing the Right Language Mode at Inference Time for Multilingual Reliabilitycited by 0
  28. 2026Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoningcited by 0
  29. 2026Cross-Lingual Alignment for Decoder-Only Models using MoE Routerscited by 0
  30. 2026Rewrite What Matters: Adaptive Multilingual Query Rewriting for Reasoning via Agentic Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MultiNRC: A Challenging and Native Multilingual Reasoning Evaluation Benchmark for LLMs
  2. 2024Eliciting Better Multilingual Structured Reasoning from LLMs through Code
  3. 2025Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoning
  4. 2026Rethinking the Multilingual Reasoning Gap with Layer Swap
  5. 2026Multilingual Reasoning Cascades Need More Context
  6. 2023On the Calibration of Multilingual Question Answering LLMs
  7. 2026mmPISA-bench: Do LLMs Reason Equally Well Across 43 Languages?
  8. 2025The Reasoning Lingua Franca: A Double-Edged Sword for Multilingual AI