MIDB: Multilingual Instruction Data Booster for Enhancing Cultural Equality in Multilingual Instruction Synthesis
On the map: Synthetic data and self-improvement for LLMs → Code and instruction data synthesis
Abstract (arXiv)
Despite doubts on data quality, instruction synthesis has been widely applied into instruction tuning (IT) of LLMs as an economic and rapid alternative. Recent endeavors focus on improving data quality for synthesized instruction pairs in English and have facilitated IT of English-centric LLMs. However, data quality issues in multilingual synthesized instruction pairs are even more severe, since the common synthesizing practice is to translate English synthesized data into other languages using machine translation (MT). Besides the known content errors in these English synthesized data, multilingual synthesized instruction data are further exposed to defects introduced by MT and face insufficient localization of the target languages, leading to cultural inequality in trained LLMs. In this paper, we propose MIDB, a Multilingual Instruction Data Booster to automatically address the quality issues in multilingual synthesized data. MIDB is trained on around 36.8k revision examples across 16 languages by human linguistic experts, thereby can boost the low-quality data by addressing content errors and MT defects, and improving localization in these synthesized data. Both automatic and human evaluation indicate that not only MIDB steadily improved instruction data quality in 16 languages, but also the instruction-following and cultural-understanding abilities of multilingual LLMs fine-tuned on MIDB-boosted data were significantly enhanced, suggesting an improved linguistic and cultural equality.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 14 of its 38 references that are in the map. First posted on arXiv 23 May 2025.
Builds on
14 of 38 references.
References · 14
- 2023GPT-4 Technical Reportcited by 18,396
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2025Qwen3 Technical Reportcited by 8,820
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 316
- 2023Extrapolating Large Language Models to Non-English by Aligning Languagescited by 69
- 2023PolyLM: An Open Source Polyglot Large Language Modelcited by 64
- 2024LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedbackcited by 51
- 2023BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Modelscited by 42
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Rethinking the Instruction Quality: LIFT is What You Need
- 2024X-Instruction: Aligning Language Model in Low-resource Languages with Self-curated Cross-lingual Instructions
- 2024M2Lingual: Enhancing Multilingual, Multi-Turn Instruction Alignment in Large Language Models
- 2024Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
- 2024Enhancing and Assessing Instruction-Following with Fine-Grained Instruction Variants
- 2025Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
- 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning
- 2023Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning