mlsanityData to 14 Oct 2026

Seed-Free Synthetic Data Generation Framework for Instruction-Tuning LLMs: A Case Study in Thai

arXiv 2411.15484 ↗First posted 23 Nov 2024cs.CLcited by 3

On the map: Synthetic data and self-improvement for LLMs → Code and instruction data synthesis

Abstract (arXiv)

We present a synthetic data approach for instruction-tuning large language models (LLMs) for low-resource languages in a data-efficient manner, specifically focusing on Thai. We identify three key properties that contribute to the effectiveness of instruction-tuning datasets: fluency, diversity, and cultural context. We propose a seed-data-free framework for generating synthetic instruction-tuning data that incorporates these essential properties. Our framework employs an LLM to generate diverse topics, retrieve relevant contexts from Wikipedia, and create instructions for various tasks, such as question answering, summarization, and conversation. The experimental results show that our best-performing synthetic dataset, which incorporates all three key properties, achieves competitive performance using only 5,000 instructions when compared to state-of-the-art Thai LLMs trained on hundreds of thousands of instructions. Our code and dataset are publicly available at https://github.com/parinzee/seed-free-synthetic-instruct.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 16 of its 34 references that are in the map. First posted on arXiv 23 Nov 2024.

Builds on

16 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec

Other references · 15

  1. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  2. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  3. 2023LIMA: Less Is More for Alignmentcited by 1,100
  4. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  5. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  6. 2023MEGA: Multilingual Evaluation of Generative AIcited by 293
  7. 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 182
  8. 2020A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languagescited by 131
  9. 2023MoDS: Model-oriented Data Selection for Instruction Tuningcited by 107
  10. 2023Don't Trust ChatGPT when Your Question is not in English: A Study of Multilingual Abilities and Types of LLMscited by 104
  11. 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
  12. 2024From LLM to NMT: Advancing Low-Resource Machine Translation with Claudecited by 58
3 more
  1. 2023CLASS: A Design Framework for building Intelligent Tutoring Systems based on Learning Science principlescited by 48
  2. 2022Quality Controlled Paraphrase Generationcited by 31
  3. 2023Adapter-TST: A Parameter Efficient Method for Multiple-Attribute Text Style Transfercited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Assessing Thai Dialect Performance in LLMs with Automatic Benchmarks and Human Evaluationcited by 5
  2. 2025SEA-BED: How Do Embedding Models Represent Southeast Asian Languages?cited by 2
  3. 2026Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation
  2. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Code
  3. 2024Balancing Cost and Effectiveness of Synthetic Data Generation Strategies for LLMs
  4. 2025Scaling Instruction-Tuned LLMs to Million-Token Contexts via Hierarchical Synthetic Data Generation
  5. 2024A Survey on Data Synthesis and Augmentation for Large Language Models
  6. 2026It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs
  7. 2024CodecLM: Aligning Language Models with Tailored Synthetic Data
  8. 2024Evaluating Language Models as Synthetic Data Generators