mlsanityData to 7 Oct 2026

Let Training Guide Selection: Online Synthetic Data Filtering via Real-Anchored Utility

arXiv 2609.29988 ↗First posted 24 Sep 2026cs.LGcited by 0

On the map: Synthetic data and self-improvement for LLMs → Model collapse from synthetic data also fits: LLM synthetic data generation

Abstract (arXiv)

Synthetic data can scale training supervision when real-world data are limited, but noise and distribution mismatch can reduce its value. Existing synthetic data selection methods often emphasize fidelity or diversity rather than the learner's evolving needs. We propose FROST, an online framework that estimates synthetic-data utility through gradient feedback anchored in real training data. It calibrates batch utility against recent history to determine when filtering is needed and filters samples only in out-of-band batches to determine what to retain, without an external verifier or held-out validation set. Experiments on two public benchmarks for image classification and LLM fine-tuning for text-to-SQL show that FROST filters out around 20--30% of the synthetic data while improving real-task performance compared with training on the full synthetic data pool. We further apply FROST during training in a large-scale industrial ads re-ranking system, achieving significant performance gains over a highly optimized production baseline, demonstrating its effectiveness and generalizability.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Model collapse from synthetic data, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 26 of its 39 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

26 of 39 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  5. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  6. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  7. 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 529
  8. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  9. 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 326
  10. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 323
  11. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  12. 2021GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Trainingcited by 269
14 more
  1. 2018GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classificationcited by 242
  2. 2023LLMRec: Large Language Models with Graph Augmentation for Recommendationcited by 171
  3. 2023Scaling Laws of Synthetic Images for Model Training ... for Nowcited by 107
  4. 2023InfoBatch: Lossless Training Speed Up by Unbiased Dynamic Data Pruningcited by 90
  5. 2025OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scalecited by 90
  6. 2023Diversity is Definitely Needed: Improving Model-Agnostic Zero-shot Classification via Stable Diffusioncited by 81
  7. 2023Explore the Power of Synthetic Data on Few-shot Object Detectioncited by 38
  8. 2020A Study of Gradient Variance in Deep Learningcited by 27
  9. 2024The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Bettercited by 19
  10. 2021Frequency-aware SGD for Efficient Embedding Learning with Provable Benefitscited by 6
  11. 2026When Pretty Isn't Useful: Investigating Why Modern Text-to-Image Models Fail as Reliable Training Data Generatorscited by 6
  12. 2025High-dimensional Analysis of Synthetic Data Selectioncited by 5
  13. 2026OrderDP: A Theoretically Guaranteed Lossless Dynamic Data Pruning Frameworkcited by 2
  14. 2026Synthetic Data from Cross-Domain Events for Large-Scale Recommendation Systemscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Training-Aware Target Coverage for Synthetic Data Selection
  2. 2025Understanding Trade offs When Conditioning Synthetic Data
  3. 2025Boosting Statistic Learning with Synthetic Data from Pretrained Large Models
  4. 2023Real-Fake: Effective Training Data Synthesis Through Distribution Matching
  5. 2024A Survey of Data Synthesis Approaches
  6. 2026Effective Synthetic Data Curation Requires Group-Level Signals
  7. 2024Multi-Armed Bandit Approach for Optimizing Training on Synthetic Data
  8. 2025Spend Wisely: Maximizing Post-Training Gains in Iterative Synthetic Data Bootstrapping