mlsanityData to 14 Oct 2026

Instruction Tuning on Public Government and Cultural Data for Low-Resource Language: a Case Study in Kazakh

arXiv 2502.13647 ↗First posted 19 Feb 2025cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Instruction tuning in low-resource languages remains underexplored due to limited text data, particularly in government and cultural domains. To address this, we introduce and open-source a large-scale (10,600 samples) instruction-following (IFT) dataset, covering key institutional and cultural knowledge relevant to Kazakhstan. Our dataset enhances LLMs' understanding of procedural, legal, and structural governance topics. We employ LLM-assisted data generation, comparing open-weight and closed-weight models for dataset construction, and select GPT-4o as the backbone. Each entity of our dataset undergoes full manual verification to ensure high quality. We also show that fine-tuning Qwen, Falcon, and Gemma on our dataset leads to consistent performance improvements in both multiple-choice and generative tasks, demonstrating the potential of LLM-assisted instruction tuning for low-resource languages.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 21 of its 50 references that are in the map. First posted on arXiv 19 Feb 2025.

Builds on

21 of 50 references.

References · 21

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  5. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  6. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  7. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  8. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  9. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  10. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 397
  11. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 311
  12. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 202
9 more
  1. 2023InstructAlign: High-and-Low Resource Language Alignment via Continual Crosslingual Instruction Tuningcited by 21
  2. 2024X-Instruction: Aligning Language Model in Low-resource Languages with Self-curated Cross-lingual Instructionscited by 18
  3. 2024CIDAR: Culturally Relevant Instruction Dataset For Arabiccited by 17
  4. 2021KazNERD: Kazakh Named Entity Recognition Datasetcited by 15
  5. 2024Mixing It Up: The Cocktail Effect of Multi-Task Fine-Tuning on LLM Performance -- A Case Study in Financecited by 15
  6. 2021KazakhTTS: An Open-Source Kazakh Text-to-Speech Synthesis Datasetcited by 13
  7. 2024MURI: High-Quality Instruction Tuning Datasets for Low-Resource Languages via Reverse Instructionscited by 12
  8. 2024KazQAD: Kazakh Open-Domain Question Answering Datasetcited by 7
  9. 2024KazSAnDRA: Kazakh Sentiment Analysis Dataset of Reviews and Attitudescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Sherkala-Chat: Building a State-of-the-Art LLM for Kazakh in a Moderately Resourced Settingcited by 4
  2. 2025LuxInstruct: A Cross-Lingual Instruction Tuning Dataset For Luxembourgishcited by 1
  3. 2025Qomhra: A Bilingual Irish and English Large Language Modelcited by 1
  4. 2026From National Curricula to Cultural Awareness: Constructing Open-Ended Culture-Specific Question Answering Datasetcited by 0