mlsanityData to 14 Oct 2026

SoftSRV: Learn to Generate Targeted Synthetic Data

arXiv 2410.16534 ↗First posted 21 Oct 2024cs.LGcited by 3

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

We present a novel framework, SoftSRV, that is used to generate targeted synthetic fine-tuning data for improving task-specific model performance. Given a sample from a target distribution, our proposed framework uses a data-driven loss minimization approach to steer a frozen large language model (LLM) to generate synthetic sequences that are similar to those from the target distribution. SoftSRV provides a practical improvement over common prompt engineering approaches that rely on human-engineered prompt-templates, which can be idiosyncratic, labor-intensive to craft, and may need to be specialized per domain. We empirically evaluate our method against standard baselines guiding a large LLM to generate synthetic data to fine-tune a smaller language model on three different domains (coding, math, reasoning). We perform these evaluations without any particular specialization of the framework to each domain, emphasizing the generality of our approach. We find that SoftSRV improves upon typical prompt engineering approaches, generating targeted data that leads to fine-tuned models with significantly better task-specific performance. In addition, SoftSRV-generated data better matches the target distribution according to the MAUVE similarity metric.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 29 of its 35 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

29 of 35 references.

References · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  6. 2021Program Synthesis with Large Language Modelscited by 3,637
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  9. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  10. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  11. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  12. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
17 more
  1. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  2. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  3. 2023Textbooks Are All You Needcited by 569
  4. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 565
  5. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  6. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 449
  7. 2023Learning to Compress Prompts with Gist Tokenscited by 390
  8. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 379
  9. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 159
  10. 2024Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Modelscited by 99
  11. 2022Language Models Can Teach Themselves to Program Bettercited by 78
  12. 2023TinyGSM: achieving >80% on GSM8k with small language modelscited by 70
  13. 2024LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancementcited by 70
  14. 2023When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitationscited by 43
  15. 2023Prompting Hard or Hardly Prompting: Prompt Inversion for Text-to-Image Diffusion Modelscited by 40
  16. 2024MEND: Meta dEmonstratioN Distillation for Efficient and Effective In-Context Learningcited by 15
  17. 2023TransCoder: Towards Unified Transferable Code Representation Learning Inspired by Human Skillscited by 8

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMscited by 16
  2. 2025ACTG-ARL: Differentially Private Conditional Text Generation with RL-Boosted Controlcited by 3
  3. 2025Better Datasets Start From RefineLab: Automatic Optimization for High-Quality Dataset Refinementcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2025BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation
  2. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Code
  3. 2024Generating Synthetic Datasets for Few-shot Prompt Tuning
  4. 2023Synthetic Data Generation in Low-Resource Settings via Fine-Tuning of Large Language Models
  5. 2025Characterizing Model Behavior Under Synthetic Data Training: An Empirical Study Across Scales and Mixing Ratios
  6. 2023TarGEN: Targeted Data Generation with Large Language Models
  7. 2025Multi-Sample Prompting and Actor-Critic Prompt Optimization for Diverse Synthetic Data Generation
  8. 2024Better Synthetic Data by Retrieving and Transforming Existing Datasets