Pragyaan: Designing and Curating High-Quality Cultural Post-Training Datasets for Indian Languages
Not on the map. Why?
Abstract (arXiv)
The effectiveness of Large Language Models (LLMs) depends heavily on the availability of high-quality post-training data, particularly instruction-tuning and preference-based examples. Existing open-source datasets, however, often lack multilingual coverage, cultural grounding, and suffer from task diversity gaps that are especially pronounced for Indian languages. We introduce a human-in-the-loop pipeline that combines translations with synthetic expansion to produce reliable and diverse Indic post-training data. Using this pipeline, we curate two datasets: Pragyaan-IT (22.5K) and Pragyaan-Align (100K) across 10 Indian languages covering 13 broad and 56 sub-categories, leveraging 57 diverse datasets. Our dataset protocol incorporates several often-overlooked dimensions and emphasize task diversity, multi-turn dialogue, instruction fidelity, safety alignment, and preservation of cultural nuance, providing a foundation for more inclusive and effective multilingual LLMs.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 47 of its 78 references that are in the map. First posted on arXiv 8 Oct 2025.
Builds on
47 of 78 references.
References · 47
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
35 more
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
- 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
- 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
- 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
- 2023LIMA: Less Is More for Alignmentcited by 1,100
- 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
- 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
- 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
- 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
- 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
- 2025LIMO: Less is More for Reasoningcited by 498
- 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 478
- 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 406
- 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 404
- 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 397
- 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 346
- 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 232
- 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
- 2025Phi-4-reasoning Technical Reportcited by 151
- 2024OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Datasetcited by 143
- 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 130
- 2025KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Codingcited by 119
- 2021Gender Bias in Machine Translationcited by 117
- 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 108
- 2022Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languagescited by 102
- 2021NaturalProofs: Mathematical Theorem Proving in Natural Languagecited by 81
- 2024IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languagescited by 56
- 2024IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languagescited by 52
- 2022IndicNLG Benchmark: Multilingual Datasets for Diverse NLG Tasks in Indic Languagescited by 42
- 2022Naamapadam: A Large-Scale Named Entity Annotated Data for Indic Languagescited by 28
- 2025FineMedLM-o1: Enhancing Medical Knowledge Reasoning Ability of LLM from Supervised Fine-Tuning to Test-Time Trainingcited by 19
- 2024L3Cube-IndicQuest: A Benchmark Question Answering Dataset for Evaluating Knowledge of LLMs in Indic Contextcited by 10
- 2023MegaWika: Millions of reports and their sources across 50 diverse languagescited by 9
- 2023Measuring Sentiment Bias in Machine Translationcited by 5
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generationcited by 1
- 2026From National Curricula to Cultural Awareness: Constructing Open-Ended Culture-Specific Question Answering Datasetcited by 0
- 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0
- 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systems18 Sep 2026
- 2026Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation13 Apr 2026
- 2026From National Curricula to Cultural Awareness: Constructing Open-Ended Culture-Specific Question Answering Dataset8 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2024Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages
- 2025HinTel-AlignBench: A Framework and Benchmark for Hindi-Telugu with English-Aligned Samples
- 2025Mind the Gap: A Review of Arabic Post-Training Datasets and Their Limitations
- 2025Llama-3-Nanda-10B-Chat: An Open Generative Large Language Model for Hindi
- 2026Inspect India Evals: An Open Benchmarking Framework for Evaluating Large Language Models in the Indian Linguistic and Cultural Context
- 2026AlignCultura: Towards Culturally Aligned Large Language Models?
- 2025BhashaKritika: Building Synthetic Pretraining Data at Scale for Indic Languages
- 2025DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models' Understanding on Indian Culture