mlsanityData to 7 Oct 2026

MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization

arXiv 2401.06838 ↗First posted 12 Jan 2024cs.CLcited by 68

On the map: LLM reasoning → Reasoning benchmarks and evaluation

also in 1 more
Abstract (arXiv)

Though reasoning abilities are considered language-agnostic, existing LLMs exhibit inconsistent reasoning abilities across different languages, e.g., reasoning in the dominant language like English is superior to other languages due to the imbalance of multilingual training data. To enhance reasoning abilities in non-dominant languages, we propose a Multilingual-Alignment-as-Preference Optimization framework (MAPO), aiming to align the reasoning processes in other languages with the dominant language. Specifically, we harness an off-the-shelf translation model for the consistency between answers in non-dominant and dominant languages, which we adopt as the preference for optimization, e.g., Direct Preference Optimization (DPO) or Proximal Policy Optimization (PPO). Experiments show that MAPO stably achieves significant improvements in the multilingual reasoning of various models on all three benchmarks (MSVAMP +16.2%, MGSM +6.1%, and MNumGLUESub +13.3%), with improved reasoning consistency across languages.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 18 of its 20 references that are in the map. First posted on arXiv 12 Jan 2024.

Builds on

18 of 20 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting11 May

Other references · 17

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  9. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  10. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  11. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  12. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
5 more
  1. 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 226
  2. 2022NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Taskscited by 110
  3. 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languagescited by 96
  4. 2023Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observationscited by 58
  5. 2023Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language Modelscited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 34; 2026: 18.

68 papers.

Sort:

By area:

  1. 2024Iterative Reasoning Preference Optimizationcited by 244
  2. 2024Aesthetic Post-Training Diffusion Models from Generic Preferences with Step-by-step Preference Optimizationcited by 81
  3. 2025NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questionscited by 81
  4. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learningcited by 62
  5. 2024M-RewardBench: Evaluating Reward Models in Multilingual Settingscited by 50
  6. 2024Self-Consistency Preference Optimizationcited by 50
  7. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 49
  8. 2025Crosslingual Reasoning through Test-Time Scalingcited by 46
  9. 2025Culture is Not Trivia: Sociocultural Theory for Cultural NLPcited by 40
  10. 2025A Survey of Direct Preference Optimizationcited by 35
  11. 2025A Survey of Multilingual Reasoning in Language Modelscited by 31
  12. 2025Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMscited by 30
38 more
  1. 2024Language Imbalance Driven Rewarding for Multilingual Self-improvingcited by 27
  2. 2024A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applicationscited by 27
  3. 2024Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Artcited by 26
  4. 2024A Survey on Human Preference Learning for Large Language Modelscited by 23
  5. 2024Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Surveycited by 23
  6. 2025Mesh-RFT: Enhancing Mesh Generation via Fine-grained Reinforcement Fine-Tuningcited by 21
  7. 2024PreAlign: Boosting Cross-Lingual Transfer by Early Establishment of Multilingual Alignmentcited by 20
  8. 2024Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Modelscited by 20
  9. 2025Language-Specific Latent Process Hinders Cross-Lingual Performancecited by 20
  10. 2024Multilingual Large Language Models: A Systematic Surveycited by 18
  11. 2025MPO: Multilingual Safety Alignment via Reward Gap Optimizationcited by 18
  12. 2024Towards a Unified View of Preference Learning for Large Language Models: A Surveycited by 17
  13. 2025SLAM: Towards Efficient Multilingual Reasoning via Selective Language Alignmentcited by 17
  14. 2025Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learningcited by 17
  15. 2025Reasoning Beyond Limits: Advances and Open Problems for LLMscited by 16
  16. 2025Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Trainingcited by 15
  17. 2025CARE: Multilingual Human Preference Learning for Cultural Awarenesscited by 15
  18. 2025Cross-lingual Collapse: How Language-Centric Foundation Models Shape Reasoning in Large Language Modelscited by 11
  19. 2025Why Do Multilingual Reasoning Gaps Emerge in Reasoning Language Models?cited by 11
  20. 2025CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answeringcited by 10
  21. 2025Aligning LLMs for Multilingual Consistency in Enterprise Applicationscited by 10
  22. 2025Implicit Cross-Lingual Rewarding for Efficient Multilingual Preference Alignmentcited by 9
  23. 2025MrGuard: A Multilingual Reasoning Guardrail for Universal LLM Safetycited by 9
  24. 2026Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignmentcited by 9
  25. 2025How Does Alignment Enhance LLMs' Multilingual Capabilities? A Language Neurons Perspectivecited by 8
  26. 2025Divergence Minimization Preference Optimization for Diffusion Model Alignmentcited by 8
  27. 2025EraseFlow: Learning Concept Erasure Policies via GFlowNet-Driven Alignmentcited by 8
  28. 2026CURE-Med: Curriculum-Informed Reinforcement Learning for Multilingual Medical Reasoningcited by 8
  29. 2024The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insightscited by 7
  30. 2025Auto-Connect: Connectivity-Preserving RigFormer with Direct Preference Optimizationcited by 7
  31. 2025Towards Better Understanding of Program-of-Thought Reasoning in Cross-Lingual and Multilingual Environmentscited by 6
  32. 2025Relic: Enhancing Reward Model Generalization for Low-Resource Indic Languages with Few-Shot Examplescited by 6
  33. 2024Why Not Transform Chat Large Language Models to Non-English?cited by 5
  34. 2026R3S: Refining and Recovering Reinforcement Signals for Multilingual Understanding and Reasoningcited by 5
  35. 2025Benchmarking and Boosting Multilingual Capabilities of LVLMs via OCR-Centric Reinforcement Learningcited by 4
  36. 2025Understanding LLMs' Cross-Lingual Context Retrieval: How Good It Is And Where It Comes Fromcited by 4
  37. 2025Souper-Model: How Simple Arithmetic Unlocks State-of-the-Art LLM Performancecited by 4
  38. 2026Cross-Lingual Activation Steering for Multilingual Language Modelscited by 4

50 most cited of 68 All 68 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Cross-lingual Self-Consistency for Multilingual Reasoning with Language Models
  2. 2025Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoning
  3. 2026TAPO: Translation Augmented Policy Optimization for Multilingual Mathematical Reasoning
  4. 2026DGPO: Beyond Pairwise Preferences with Directional Consistent Groupwise Optimization
  5. 2025Cross-Lingual Consistency: A Novel Inference Framework for Advancing Reasoning in Large Language Models
  6. 2024LinguaLIFT: An Effective Two-stage Instruction Tuning Framework for Low-Resource Language Reasoning
  7. 2026AdaMame: A Training Recipe for Adaptive Multilingual Reasoning
  8. 2026LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance