mlsanityData to 14 Oct 2026

Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptation

arXiv 2305.15011 ↗First posted 24 May 2023cs.CLcited by 81

Not on the map. Why?

Abstract (arXiv)

Instruction tuning has shown great promise in improving the performance of large language models. However, research on multilingual instruction tuning has been limited due to the scarcity of high-quality instruction-response datasets across different languages. To bridge this gap, we present Bactrian-X, a comprehensive multilingual parallel dataset of 3.4 million instruction-response pairs across 52 languages. Leveraging this dataset, we train a set of adapters using low-rank adaptation (LoRA), which are lightweight components that seamlessly integrate with large language models. These adapters have a substantially lower parameter count than the base model, making them easily replaceable and usable as plug-ins for different languages or language groups. Extensive experiments in various multilingual evaluation settings demonstrate that models derived from LoRA-based training over Bactrian-X outperform both the vanilla models and existing instruction-tuned models. The code and models are publicly available at https://github.com/mbzuai-nlp/bactrian-x

Key facts. Cited by 81 papers in the map (data through 14 Oct 2026). Builds on 40 of its 58 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

40 of 58 references.

References · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  8. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  9. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  11. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  12. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
28 more
  1. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  3. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  4. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  5. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  6. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  7. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  8. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  9. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  10. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  11. 2019How multilingual is Multilingual BERT?cited by 645
  12. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 417
  13. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 352
  14. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 338
  15. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 333
  16. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
  17. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  18. 2019Massively Multilingual Neural Machine Translationcited by 246
  19. 2023LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructionscited by 124
  20. 2020BLEU might be Guilty but References are not Innocentcited by 110
  21. 2021It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoningcited by 69
  22. 2022NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languagescited by 68
  23. 2022One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesiacited by 66
  24. 2020EXAMS: A Multi-Subject High School Examinations Dataset for Cross-Lingual and Multilingual Question Answeringcited by 60
  25. 2020IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLPcited by 58
  26. 2023SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)cited by 41
  27. 2021Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Trainingcited by 19
  28. 2022Universal Conditional Masked Language Pre-training for Neural Machine Translationcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 16; 2024: 50; 2025: 7; 2026: 8.

81 papers.

Sort:

By area:

  1. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 449
  2. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 295
  3. 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 190
  4. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 177
  5. 2023Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integrationcited by 172
  6. 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 123
  7. 2024LLaMA Beyond English: An Empirical Study on Language Capability Transfercited by 95
  8. 2024The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contextscited by 94
  9. 2023Style Over Substance: Evaluation Biases for Large Language Modelscited by 70
  10. 2023Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Modelscited by 69
  11. 2023Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpacacited by 60
  12. 2024Adapting Large Language Models for Document-Level Machine Translationcited by 59
38 more
  1. 2023Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLUcited by 53
  2. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  3. 2024LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedbackcited by 51
  4. 2024(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Textscited by 49
  5. 2024SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languagescited by 43
  6. 2024EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Modelscited by 41
  7. 2024IndoCulture: Exploring Geographically-Influenced Cultural Commonsense Reasoning Across Eleven Indonesian Provincescited by 39
  8. 2023COPAL-ID: Indonesian Language Reasoning with Local Culture and Nuancescited by 35
  9. 2023Improving In-context Learning of Multilingual Generative Language Models with Cross-lingual Alignmentcited by 33
  10. 2023Aligning Large Language Models with Human: A Surveycited by 32
  11. 2023Are Multilingual LLMs Culturally-Diverse Reasoners? An Investigation into Multicultural Proverbs and Sayingscited by 32
  12. 2024Airavata: Introducing Hindi Instruction-tuned LLMcited by 32
  13. 2025The Bitter Lesson Learned from 2,000+ Multilingual Benchmarkscited by 31
  14. 2024Language Imbalance Driven Rewarding for Multilingual Self-improvingcited by 28
  15. 2025Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoningcited by 24
  16. 2024Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Surveycited by 23
  17. 2024Fine-tuning Large Language Models with Sequential Instructionscited by 20
  18. 2023TaCo: Enhancing Cross-Lingual Transfer for Low-Resource Languages in LLMs through Translation-Assisted Chain-of-Thought Processescited by 19
  19. 2024Beyond Probabilities: Unveiling the Misalignment in Evaluating Large Language Modelscited by 19
  20. 2024Unlocking the Potential of Model Merging for Low-Resource Languagescited by 19
  21. 2024X-Instruction: Aligning Language Model in Low-resource Languages with Self-curated Cross-lingual Instructionscited by 18
  22. 2024Multilingual Large Language Models: A Systematic Surveycited by 18
  23. 2024CIDAR: Culturally Relevant Instruction Dataset For Arabiccited by 17
  24. 2025New Trends for Modern Machine Translation with Large Reasoning Modelscited by 17
  25. 2024Mixture-of-Skills: Learning to Optimize Data Usage for Fine-Tuning Large Language Modelscited by 16
  26. 2024METAL: Towards Multilingual Meta-Evaluationcited by 15
  27. 2024TeenyTinyLlama: open-source tiny language models trained in Brazilian Portuguesecited by 13
  28. 2024How Reliable Are Automatic Evaluation Methods for Instruction-Tuned LLMs?cited by 13
  29. 2024mEdIT: Multilingual Text Editing via Instruction Tuningcited by 13
  30. 2024Tucano: Advancing Neural Text Generation for Portuguesecited by 13
  31. 2024Vikhr: The Family of Open-Source Instruction-Tuned Large Language Models for Russiancited by 12
  32. 2024MURI: High-Quality Instruction Tuning Datasets for Low-Resource Languages via Reverse Instructionscited by 12
  33. 2026F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual Worldcited by 12
  34. 2024Zero-shot cross-lingual transfer in instruction tuning of large language modelscited by 11
  35. 2024Analyzing and Adapting Large Language Models for Few-Shot Multilingual NLU: Are We There Yet?cited by 11
  36. 2024Investigating Multilingual Instruction-Tuning: Do Polyglot Models Demand for Multilingual Instructions?cited by 10
  37. 2024Komodo: A Linguistic Expedition into Indonesia's Regional Languagescited by 10
  38. 2025Llama-3-Nanda-10B-Chat: An Open Generative Large Language Model for Hindicited by 10

50 most cited of 81 All 81 (JSON)