# Seed-Free Synthetic Data Generation Framework for Instruction-Tuning LLMs: A Case Study in Thai

arXiv: https://arxiv.org/abs/2411.15484 · HTML: https://mlsanity.com/paper/2411.15484/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 16 of its 34 references that are in the map.
- First posted on arXiv 23 Nov 2024.

## Placed in

- Synthetic data and self-improvement for LLMs / [Code and instruction data synthesis](https://mlsanity.com/synth/code-and-instruction-data-synthesis/)

## Builds on (most cited first, 16 of 16)

- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Enhancing Chat Language Models by Scaling High-quality Instructional Conversations](https://mlsanity.com/paper/2305.14233/) (2023)
- [MEGA: Multilingual Evaluation of Generative AI](https://mlsanity.com/paper/2303.12528/) (2023)
- [A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity](https://mlsanity.com/paper/2305.13169/) (2023)
- [A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages](https://mlsanity.com/paper/2006.06202/) (2020)
- [MoDS: Model-oriented Data Selection for Instruction Tuning](https://mlsanity.com/paper/2311.15653/) (2023)
- [Don't Trust ChatGPT when Your Question is not in English: A Study of Multilingual Abilities and Types of LLMs](https://mlsanity.com/paper/2305.16339/) (2023)
- [SeaLLMs -- Large Language Models for Southeast Asia](https://mlsanity.com/paper/2312.00738/) (2023)
- [From LLM to NMT: Advancing Low-Resource Machine Translation with Claude](https://mlsanity.com/paper/2404.13813/) (2024)
- [CLASS: A Design Framework for building Intelligent Tutoring Systems based on Learning Science principles](https://mlsanity.com/paper/2305.13272/) (2023)
- [Quality Controlled Paraphrase Generation](https://mlsanity.com/paper/2203.10940/) (2022)
- [Adapter-TST: A Parameter Efficient Method for Multiple-Attribute Text Style Transfer](https://mlsanity.com/paper/2305.05945/) (2023)

## Built on by (most cited first, 3 of 3)

- [Assessing Thai Dialect Performance in LLMs with Automatic Benchmarks and Human Evaluation](https://mlsanity.com/paper/2504.05898/) (2025)
- [SEA-BED: How Do Embedding Models Represent Southeast Asian Languages?](https://mlsanity.com/paper/2508.12243/) (2025)
- [Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation](https://mlsanity.com/paper/2604.11290/) (2026)

## Abstract (arXiv)

We present a synthetic data approach for instruction-tuning large language models (LLMs) for low-resource languages in a data-efficient manner, specifically focusing on Thai. We identify three key properties that contribute to the effectiveness of instruction-tuning datasets: fluency, diversity, and cultural context. We propose a seed-data-free framework for generating synthetic instruction-tuning data that incorporates these essential properties. Our framework employs an LLM to generate diverse topics, retrieve relevant contexts from Wikipedia, and create instructions for various tasks, such as question answering, summarization, and conversation. The experimental results show that our best-performing synthetic dataset, which incorporates all three key properties, achieves competitive performance using only 5,000 instructions when compared to state-of-the-art Thai LLMs trained on hundreds of thousands of instructions. Our code and dataset are publicly available at https://github.com/parinzee/seed-free-synthetic-instruct.