mlsanityData to 14 Oct 2026

MindMerger: Efficient Boosting LLM Reasoning in non-English Languages

arXiv 2405.17386 ↗First posted 27 May 2024cs.CL cs.AIcited by 21

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Reasoning capabilities are crucial for Large Language Models (LLMs), yet a notable gap exists between English and non-English languages. To bridge this disparity, some works fine-tune LLMs to relearn reasoning capabilities in non-English languages, while others replace non-English inputs with an external model's outputs such as English translation text to circumvent the challenge of LLM understanding non-English. Unfortunately, these methods often underutilize the built-in skilled reasoning and useful language understanding capabilities of LLMs. In order to better utilize the minds of reasoning and language understanding in LLMs, we propose a new method, namely MindMerger, which merges LLMs with the external language understanding capabilities from multilingual models to boost the multilingual reasoning performance. Furthermore, a two-step training scheme is introduced to first train to embeded the external capabilities into LLMs and then train the collaborative utilization of the external capabilities and the built-in capabilities in LLMs. Experiments on three multilingual reasoning datasets and a language understanding dataset demonstrate that MindMerger consistently outperforms all baselines, especially in low-resource languages. Without updating the parameters of LLMs, the average accuracy improved by 6.7% and 8.0% across all languages and low-resource languages on the MGSM dataset, respectively.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 18 of its 38 references that are in the map. First posted on arXiv 27 May 2024.

Builds on

18 of 38 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024LangBridge: Multilingual Reasoning Without Multilingual Supervision19 Jan
  2. 2024Question Translation Training for Better Multilingual Reasoning15 Jan
  3. 2024xCoT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning13 Jan
  4. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting11 May

Other references · 14

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2023Mistral 7Bcited by 3,164
  6. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  7. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  8. 2024Large Language Models: A Surveycited by 525
  9. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 427
  10. 2022mGPT: Few-Shot Learners Go Multilingualcited by 142
  11. 2021Think you have Solved Direct-Answer Question Answering? Try ARC-DA, the Direct-Answer AI2 Reasoning Challengecited by 80
  12. 2023Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observationscited by 59
2 more
  1. 2023Forward-Backward Reasoning in Large Language Models for Mathematical Verificationcited by 40
  2. 2024LLM Augmented LLMs: Expanding Capabilities through Compositioncited by 35

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 11; 2026: 3.

21 papers.

Sort:

By area:

  1. 2024LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languagescited by 48
  2. 2025Crosslingual Reasoning through Test-Time Scalingcited by 46
  3. 2025A Survey of Multilingual Reasoning in Language Modelscited by 33
  4. 2024Data Contamination Can Cross Language Barrierscited by 26
  5. 2024Assessing Dialect Fairness and Robustness of Large Language Models in Reasoning Taskscited by 22
  6. 2025Language-Specific Latent Process Hinders Cross-Lingual Performancecited by 21
  7. 2024Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Mergingcited by 19
  8. 2025SLAM: Towards Efficient Multilingual Reasoning via Selective Language Alignmentcited by 18
  9. 2025Improving Multilingual Language Models by Aligning Representations through Steeringcited by 11
  10. 2025Unveiling Over-Memorization in Finetuning LLMs for Reasoning Taskscited by 11
  11. 2025LayAlign: Enhancing Multilingual Reasoning in Large Language Models via Layer-Wise Adaptive Fusion and Alignment Strategycited by 10
  12. 2024Varying Shades of Wrong: Aligning LLMs with Wrong Answers Onlycited by 9
9 more
  1. 2024Lens: Rethinking Multilingual Enhancement for Large Language Modelscited by 6
  2. 2026Gained in Translation: Privileged Pairwise Judges Enhance Multilingual Reasoningcited by 6
  3. 2025Language Specific Knowledge: Do Models Know Better in X than in English?cited by 4
  4. 2026Cross-lingual Self-Consistency for Multilingual Reasoning with Language Modelscited by 3
  5. 2025MERLIN: Multi-Stage Curriculum Alignment for Multilingual Encoder-LLM Integration in Cross-Lingual Reasoningcited by 2
  6. 2024Extracting and Combining Abilities For Building Multi-lingual Ability-enhanced Large Language Modelscited by 1
  7. 2025Utilizing Multilingual Encoders to Improve Large Language Models for Low-Resource Languagescited by 0
  8. 2025OptAgent: Optimizing Query Rewriting for E-commerce via Multi-Agent Simulationcited by 0
  9. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025EMCEE: Improving Multilingual Capability of LLMs via Bridging Knowledge and Reasoning with Extracted Synthetic Multilingual Context
  2. 2024Eliciting Better Multilingual Structured Reasoning from LLMs through Code
  3. 2025Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineering
  4. 2025Reasoning Beyond Limits: Advances and Open Problems for LLMs
  5. 2025Adapting Language-Specific LLMs to a Reasoning Model in One Day via Model Merging -- An Open Recipe
  6. 2025Cross-Lingual Consistency: A Novel Inference Framework for Advancing Reasoning in Large Language Models
  7. 2026Rethinking the Multilingual Reasoning Gap with Layer Swap
  8. 2025Could Thinking Multilingually Empower LLM Reasoning?