# GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation

arXiv: https://arxiv.org/abs/2505.20416 · HTML: https://mlsanity.com/paper/2505.20416/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Synthetic reasoning data and bootstrapping, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 26 May 2025.

## Placed in

- Synthetic data and self-improvement for LLMs / [Synthetic reasoning data and bootstrapping](https://mlsanity.com/synth/synthetic-reasoning-data-and-bootstrapping/)

## Builds on (most cited first, 23 of 23)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Symbolic Knowledge Distillation: from General Language Models to Commonsense Models](https://mlsanity.com/paper/2110.07178/) (2021)
- [Towards a Unified Multi-Dimensional Evaluator for Text Generation](https://mlsanity.com/paper/2210.07197/) (2022)
- [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://mlsanity.com/paper/2406.15126/) (2024)
- [Learning to Ask: Neural Question Generation for Reading Comprehension](https://mlsanity.com/paper/1705.00106/) (2017)
- [Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling](https://mlsanity.com/paper/2401.16380/) (2024)
- [Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions](https://mlsanity.com/paper/2411.14405/) (2024)
- [A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'](https://mlsanity.com/paper/2410.12954/) (2024)
- [Data Recombination for Neural Semantic Parsing](https://mlsanity.com/paper/1606.03622/) (2016)
- [Best Practices and Lessons Learned on Synthetic Data](https://mlsanity.com/paper/2404.07503/) (2024)
- [The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities](https://mlsanity.com/paper/2408.13296/) (2024)
- [LongForm: Effective Instruction Tuning with Reverse Instructions](https://mlsanity.com/paper/2304.08460/) (2023)
- [Genie: Achieving Human Parity in Content-Grounded Datasets Generation](https://mlsanity.com/paper/2401.14367/) (2024)
- [Prompting Large Language Models with Chain-of-Thought for Few-Shot Knowledge Base Question Generation](https://mlsanity.com/paper/2310.08395/) (2023)
- [Knowledge Questions from Knowledge Graphs](https://mlsanity.com/paper/1610.09935/) (2016)
- [SeedBench: A Multi-task Benchmark for Evaluating Large Language Models in Seed Science](https://mlsanity.com/paper/2505.13220/) (2025)
- [Self-QA: Unsupervised Knowledge Guided Language Model Alignment](https://mlsanity.com/paper/2305.11952/) (2023)
- [In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided Search](https://mlsanity.com/paper/2311.07237/) (2023)
- [SGSH: Stimulate Large Language Models with Skeleton Heuristics for Knowledge Base Question Generation](https://mlsanity.com/paper/2404.01923/) (2024)
- [How do you know that? Teaching Generative Language Models to Reference Answers to Biomedical Questions](https://mlsanity.com/paper/2407.05015/) (2024)

## Built on by (most cited first, 8 of 8)

- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [MedXIAOHE: A Comprehensive Recipe for Building Medical MLLMs](https://mlsanity.com/paper/2602.12705/) (2026)
- [TeleCom-Bench: How Far Are Large Language Models from Industrial Telecommunication Applications?](https://mlsanity.com/paper/2605.18025/) (2026)
- [SPA: A Simple but Tough-to-Beat Baseline for Knowledge Injection](https://mlsanity.com/paper/2603.22213/) (2026)
- [UniShield: Unified Face Attack Detection via KG-Informed Multimodal Reasoning](https://mlsanity.com/paper/2605.08709/) (2026)
- [Data-Centric Post-Training for Financial Reasoning: Mining, Distillation, and Verifiable Learning](https://mlsanity.com/paper/2609.10113/) (2026)
- [Repair Before Reinforce: Context-Augmented Knowledge Graph Reasoning for Multi-Hop Question Answering](https://mlsanity.com/paper/2609.12230/) (2026)
- [SAGE: Semantic Anchor-Guided Evolution for Grounded Medical QA Data Synthesis](https://mlsanity.com/paper/2610.08093/) (2026)

## Abstract (arXiv)

Fine-tuning for large language models (LLMs) typically requires substantial amounts of high-quality supervised data, which is both costly and labor-intensive to acquire. While synthetic data generation has emerged as a promising solution, existing approaches frequently suffer from factual inaccuracies, insufficient long-tail coverage, simplistic knowledge structures, and homogenized outputs. To address these challenges, we introduce GraphGen, a knowledge graph-guided framework designed for three key question-answering (QA) scenarios: atomic QA, aggregated QA, and multi-hop QA. It begins by constructing a fine-grained knowledge graph from the source text. It then identifies knowledge gaps in LLMs using the expected calibration error metric, prioritizing the generation of QA pairs that target high-value, long-tail knowledge. Furthermore, GraphGen incorporates multi-hop neighborhood sampling to capture complex relational information and employs style-controlled generation to diversify the resulting QA data. Experimental results on knowledge-intensive tasks under closed-book settings demonstrate that GraphGen outperforms conventional synthetic data methods, offering a more reliable and comprehensive solution to the data scarcity challenge in supervised fine-tuning. The code and data are publicly available at https://github.com/open-sciencelab/GraphGen.