mlsanityData to 14 Oct 2026

XL-Suite: Cross-Lingual Synthetic Training and Evaluation Data for Open-Ended Generation

arXiv 2503.22973 ↗First posted 29 Mar 2025cs.CL cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Cross-lingual open-ended generation - responding in a language different from that of the query - is an important yet understudied problem. This work proposes XL-Instruct, a novel technique for generating high-quality synthetic data, and introduces XL-AlpacaEval, a new benchmark for evaluating cross-lingual generation capabilities of large language models (LLMs). Our experiments show that fine-tuning with just 8K instructions generated using XL-Instruct significantly improves model performance, increasing the win rate against GPT-4o-Mini from 7.4% to 21.5% and improving on several fine-grained quality metrics. Moreover, base LLMs fine-tuned on XL-Instruct exhibit strong zero-shot improvements to question answering in the same language, as shown on our machine-translated m-AlpacaEval. These consistent gains highlight the promising role of XL-Instruct in the post-training of multilingual LLMs. Finally, we publicly release XL-Suite, a collection of training and evaluation data to facilitate research in cross-lingual open-ended generation.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 39 of its 55 references that are in the map. First posted on arXiv 29 Mar 2025.

Builds on

39 of 55 references.

References · 39

  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  2. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  4. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  5. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  6. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  7. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  8. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 575
  9. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
  10. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
  11. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 397
  12. 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
27 more
  1. 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 190
  2. 2023CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languagescited by 174
  3. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Promptingcited by 165
  4. 2023GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4cited by 146
  5. 2020A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languagescited by 131
  6. 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 123
  7. 2024A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'cited by 115
  8. 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languagescited by 97
  9. 2023Scaling up COMETKIWI: Unbabel-IST 2023 Submission for the Quality Estimation Shared Taskcited by 88
  10. 2024Understanding and Mitigating Language Confusion in LLMscited by 75
  11. 2023Multilingual LLMs are Better Cross-lingual In-context Learners with Alignmentcited by 71
  12. 2023Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpacacited by 60
  13. 2024M-RewardBench: Evaluating Reward Models in Multilingual Settingscited by 50
  14. 2023LongForm: Effective Instruction Tuning with Reverse Instructionscited by 47
  15. 2024UMBRELA: UMbrela is the (Open-Source Reproduction of the) Bing RELevance Assessorcited by 42
  16. 2024Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence Segmentationcited by 41
  17. 2023PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuningcited by 39
  18. 2024MM-Eval: A Multilingual Meta-Evaluation Benchmark for LLM-as-a-Judge and Reward Modelscited by 35
  19. 2024Fine-tuning Large Language Models with Sequential Instructionscited by 20
  20. 2024X-Instruction: Aligning Language Model in Low-resource Languages with Self-curated Cross-lingual Instructionscited by 18
  21. 2023DoG-Instruct: Towards Premium Instruction-Tuning Data via Text-Grounded Instruction Wrappingcited by 16
  22. 2024Quality or Quantity? On Data Scale and Diversity in Adapting Large Language Models for Low-Resource Translationcited by 13
  23. 2024MURI: High-Quality Instruction Tuning Datasets for Low-Resource Languages via Reverse Instructionscited by 12
  24. 2024Large Language Models are Good Multi-lingual Learners : When LLMs Meet Cross-lingual Promptscited by 10
  25. 2023Harnessing the Power of David against Goliath: Exploring Instruction Data Generation without Using Closed-Source Modelscited by 6
  26. 2024A Three-Pronged Approach to Cross-Lingual Adaptation with Multilingual LLMscited by 6
  27. 2024Think Carefully and Check Again! Meta-Generation Unlocking LLMs for Low-Resource Cross-Lingual Summarizationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026IRIS: Interleaved Reinforcement with Incremental Staged Curriculum for Cross-Lingual Mathematical Reasoningcited by 0