mlsanityData to 14 Oct 2026

UPDESH: Synthesizing Grounded Instruction Tuning Data for 13 Indic Languages

arXiv 2509.21294 ↗First posted 25 Sep 2025cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Developing culturally grounded multilingual AI systems remains challenging, particularly for low-resource languages. While synthetic data offers promise, its effectiveness in multilingual and multicultural contexts is underexplored. We investigate bottom-up synthetic data generation using large open-source LLMs (>= 235B parameters) grounded in language-specific Wikipedia content, complementing dominant top-down translation-based approaches from English. We introduce Updesh, a high-quality large-scale synthetic instruction-following dataset comprising 9.5M data points across 13 Indian languages and English, encompassing diverse reasoning and generative tasks. Comprehensive evaluation using automated metrics and 10K human assessments confirms high data quality. Downstream evaluations performed by fine-tuning models on various datasets and assessing performance across 13 diverse multilingual datasets and model comparative evaluations, demonstrate that models trained on Updesh consistently obtain significant improvements on NLU, NLG evaluations. Finally, through ablation studies and cultural evaluations, we show that context-aware, culturally grounded data generation is essential for effective multilingual AI development.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 40 of its 72 references that are in the map. First posted on arXiv 25 Sep 2025.

Builds on

40 of 72 references.

References · 40

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  5. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  6. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  7. 2023LIMA: Less Is More for Alignmentcited by 1,100
  8. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  9. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  10. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  11. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 311
  12. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 256
28 more
  1. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 235
  2. 2024Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluationcited by 191
  3. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 159
  4. 2023IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languagescited by 152
  5. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 103
  6. 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 89
  7. 2023Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?cited by 88
  8. 2024INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledgecited by 86
  9. 2024Understanding and Mitigating Language Confusion in LLMscited by 75
  10. 2024Multilingual Instruction Tuning With Just a Pinch of Multilingualitycited by 73
  11. 2021Machine Translationese: Effects of Algorithmic Bias on Linguistic Complexity in Machine Translationcited by 70
  12. 2023Elo Uncovered: Robustness and Best Practices in Language Model Evaluationcited by 61
  13. 2023Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpacacited by 60
  14. 2024IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languagescited by 56
  15. 2023Benchmarking Machine Translation with Cultural Awarenesscited by 53
  16. 2024IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languagescited by 52
  17. 2019The Effect of Translationese in Machine Translation Test Setscited by 49
  18. 2024PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Datacited by 37
  19. 2025The Bitter Lesson Learned from 2,000+ Multilingual Benchmarkscited by 31
  20. 2024Better Alignment with Instruction Back-and-Forth Translationcited by 16
  21. 2023Bhasha-Abhijnaanam: Native-script and romanized Language Identification for 22 Indic languagescited by 13
  22. 2024Case2Code: Scalable Synthetic Data for Code Generationcited by 13
  23. 2025Quantifying Language Disparities in Multilingual Large Language Modelscited by 7
  24. 2024Towards Building Large Scale Datasets and State-of-the-Art Automatic Speech Translation Systems for 14 Indian Languagescited by 5
  25. 2025DCAD-2000: A Multilingual Dataset across 2000+ Languages with Data Cleaning as Anomaly Detectioncited by 5
  26. 2024Towards Inducing Long-Context Abilities in Multilingual Neural Machine Translation Modelscited by 3
  27. 2025D'ej`a Vu: Multilingual LLM Evaluation through the Lens of Machine Translation Evaluationcited by 3
  28. 2025Evaluating the Effectiveness and Scalability of LLM-Based Data Augmentation for Retrievalcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Indi-RomCoM: Code-Mixed Benchmark for Evaluating LLMs on Romanized Indic-English Instructionscited by 0
  2. 2026PSK at WMT 2026 MIST: Task-Specialized QLoRA Adapters for Multilingual Summarization and Question Answeringcited by 0
  3. 2026Cross-Lingual Alignment for Decoder-Only Models using MoE Routerscited by 0
  4. 2026Measuring Cultural Alignment Beyond the Average: A Framework for Evaluating Maternal-Health LLM Interactions in Indian Contextscited by 0