mlsanityData to 14 Oct 2026

BayLing 2: A Multilingual Large Language Model with Efficient Language Alignment

arXiv 2411.16300 ↗First posted 25 Nov 2024cs.CL cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs), with their powerful generative capabilities and vast knowledge, empower various tasks in everyday life. However, these abilities are primarily concentrated in high-resource languages, leaving low-resource languages with weaker generative capabilities and relatively limited knowledge. Enhancing the multilingual capabilities of LLMs is therefore crucial for serving over 100 linguistic communities worldwide. An intuitive approach to enhance the multilingual capabilities would be to construct instruction data for various languages, but constructing instruction data for over 100 languages is prohibitively costly. In this paper, we introduce BayLing 2, which efficiently transfers generative capabilities and knowledge from high-resource languages to low-resource languages through language alignment. To achieve this, we constructed a dataset of 3.2 million instructions, comprising high-resource language instructions (Chinese and English) and cross-lingual instructions for 100+ languages and performed instruction tuning based on the dataset to facilitate the capability transfer between languages. Using Llama as the foundation model, we developed BayLing-2-7B, BayLing-2-13B, and BayLing-2-8B, and conducted a comprehensive evaluation of BayLing. For multilingual translation across 100+ languages, BayLing shows superior performance compared to open-source models of similar scale. For multilingual knowledge and understanding benchmarks, BayLing achieves significant improvements across over 20 low-resource languages, demonstrating its capability of effective knowledge transfer from high-resource to low-resource languages. Furthermore, results on English benchmarks indicate that BayLing maintains high performance in highresource languages while enhancing the performance in low-resource languages. Demo, homepage, code and models of BayLing are available.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 22 of its 34 references that are in the map. First posted on arXiv 25 Nov 2024.

Builds on

22 of 34 references.

References · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  5. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  6. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  7. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  8. 2016Training Deep Nets with Sublinear Memory Costcited by 848
  9. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 807
  10. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  11. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 311
  12. 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 190
10 more
  1. 2023CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languagescited by 174
  2. 2022Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuningcited by 149
  3. 2023Continual Pre-Training of Large Language Models: How to (re)warm your model?cited by 137
  4. 2020EXAMS: A Multi-Subject High School Examinations Dataset for Cross-Lingual and Multilingual Question Answeringcited by 60
  5. 2023BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Modelscited by 42
  6. 2024Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languagescited by 38
  7. 2019MultiFiT: Efficient Multi-lingual Language Model Fine-tuningcited by 31
  8. 2022GreenPLM: Cross-Lingual Transfer of Monolingual Pre-Trained Language Models at Almost No Costcited by 24
  9. 2023Can Large Language Model Comprehend Ancient Chinese? A Preliminary Test on ACLUEcited by 10
  10. 2023How Good are Commercial Large Language Models on African Languages?cited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 3.

8 papers.

  1. 2025Less, but Better: Efficient Multilingual Expansion for LLMs via Layer-wise Mixture-of-Expertscited by 12
  2. 2026Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignmentcited by 9
  3. 2025AlignX: Advancing Multilingual Large Language Models with Multilingual Representation Alignmentcited by 6
  4. 2025MiLiC-Eval: Benchmarking Multilingual LLMs for China's Minority Languagescited by 5
  5. 2025Document Intelligence in the Era of Large Language Models: A Surveycited by 5
  6. 2025IG-Pruning: Input-Guided Block Pruning for Large Language Modelscited by 3
  7. 2026Language on Demand, Knowledge at Core: Composing LLMs with Encoder-Decoder Translation Models for Extensible Multilingualitycited by 2
  8. 2026TaxPraBen: A Scalable Benchmark for Structured Evaluation of LLMs in Chinese Real-World Tax Practicecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Extrapolating Large Language Models to Non-English by Aligning Languages
  2. 2024LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback
  3. 2024M2Lingual: Enhancing Multilingual, Multi-Turn Instruction Alignment in Large Language Models
  4. 2024Responsible Multilingual Large Language Models: A Survey of Development, Applications, and Societal Impact
  5. 2024Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners
  6. 2024FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data
  7. 2023PolyLM: An Open Source Polyglot Large Language Model
  8. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias