mlsanityData to 7 Oct 2026

LayAlign: Enhancing Multilingual Reasoning in Large Language Models via Layer-Wise Adaptive Fusion and Alignment Strategy

arXiv 2502.11405 ↗First posted 17 Feb 2025cs.CLcited by 10

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Despite being pretrained on multilingual corpora, large language models (LLMs) exhibit suboptimal performance on low-resource languages. Recent approaches have leveraged multilingual encoders alongside LLMs by introducing trainable parameters connecting the two models. However, these methods typically focus on the encoder's output, overlooking valuable information from other layers. We propose aname (mname), a framework that integrates representations from all encoder layers, coupled with the attaname mechanism to enable layer-wise interaction between the LLM and the multilingual encoder. Extensive experiments on multilingual reasoning tasks, along with analyses of learned representations, show that our approach consistently outperforms existing baselines.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 13 of its 34 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

13 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024MindMerger: Efficient Boosting LLM Reasoning in non-English Languages27 May

Other references · 12

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2023Qwen Technical Reportcited by 3,273
  3. 2024Qwen2 Technical Reportcited by 2,076
  4. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  5. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  6. 2024Orca-Math: Unlocking the potential of SLMs in Grade School Mathcited by 151
  7. 2022mGPT: Few-Shot Learners Go Multilingualcited by 141
  8. 2023Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observationscited by 58
  9. 2024LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languagescited by 47
  10. 2024EVLM: An Efficient Vision-Language Model for Visual Understandingcited by 23
  11. 2024ConvoSense: Overcoming Monotonous Commonsense Inferences for Conversational AIcited by 4
  12. 2020Rethinking and Improving Natural Language Generation with Layer-Wise Multi-View Decodingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 8.

10 papers.

  1. 2026Understanding Multilingualism in Mixture-of-Experts LLMs: Routing Mechanism, Expert Specialization, and Layerwise Steeringcited by 8
  2. 2025Less Data Less Tokens: Multilingual Unification Learning for Efficient Test-Time Reasoning in LLMscited by 3
  3. 2025MERLIN: Multi-Stage Curriculum Alignment for Multilingual Encoder-LLM Integration in Cross-Lingual Reasoningcited by 2
  4. 2026Language on Demand, Knowledge at Core: Composing LLMs with Encoder-Decoder Translation Models for Extensible Multilingualitycited by 2
  5. 2026SAT: Balancing Reasoning Accuracy and Efficiency with Stepwise Adaptive Thinkingcited by 2
  6. 2026GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Modelscited by 0
  7. 2026Single-Pass, Depth-Selective Reading for Multi-Aspect Sentiment Analysiscited by 0
  8. 2026Enhancing Multilingual Reasoning via Steerable Model Mergingcited by 0
  9. 2026VFA: Empowering Multilingual MLLMs via Vision-Free Adaptationcited by 0
  10. 2026Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineering
  2. 2025Cross-Lingual Consistency: A Novel Inference Framework for Advancing Reasoning in Large Language Models
  3. 2026Soft Token Alignment for Cross-Lingual Reasoning
  4. 2025SLAM: Towards Efficient Multilingual Reasoning via Selective Language Alignment
  5. 2025Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs
  6. 2025Gradual Code-Switching as Inference-Time Cross-Lingual Representational Alignment for LLMs
  7. 2026Rethinking the Multilingual Reasoning Gap with Layer Swap
  8. 2026Maistros: A Greek Large Language Model Adapted Through Knowledge Distillation From Large Reasoning Models