# Make LLM Learn to Synthesize from Streaming Experiences through Feedback

arXiv: https://arxiv.org/abs/2605.29940 · HTML: https://mlsanity.com/paper/2605.29940/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 28 May 2026.

## Builds on (most cited first, 24 of 24)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [TinyStories: How Small Can Language Models Be and Still Speak Coherent English?](https://mlsanity.com/paper/2305.07759/) (2023)
- [Continual Learning of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2404.16789/) (2024)
- [Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias](https://mlsanity.com/paper/2306.15895/) (2023)
- [Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves](https://mlsanity.com/paper/2311.04205/) (2023)
- [InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions](https://mlsanity.com/paper/2403.11435/) (2024)
- [MathSmith: Towards Extremely Hard Mathematical Reasoning by Forging Synthetic Problems with a Reinforced Policy](https://mlsanity.com/paper/2508.05592/) (2025)
- [Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning](https://mlsanity.com/paper/2509.08255/) (2025)
- [Continual Gradient Low-Rank Projection Fine-Tuning for LLMs](https://mlsanity.com/paper/2507.02503/) (2025)
- [Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence](https://mlsanity.com/paper/2505.16694/) (2025)
- [QuitoBench: A High-Quality Open Time Series Forecasting Benchmark](https://mlsanity.com/paper/2603.26017/) (2026)
- [Teaching LLMs How to Learn with Contextual Fine-Tuning](https://mlsanity.com/paper/2503.09032/) (2025)
- [Experience Replay Addresses Loss of Plasticity in Continual Learning](https://mlsanity.com/paper/2503.20018/) (2025)
- [Thought Purity: A Defense Framework For Chain-of-Thought Attack](https://mlsanity.com/paper/2507.12314/) (2025)
- [Aligning Large Language Models via Fully Self-Synthetic Data](https://mlsanity.com/paper/2510.06652/) (2025)
- [MetaLoRA: Tensor-Enhanced Adaptive Low-Rank Fine-tuning](https://mlsanity.com/paper/2504.00460/) (2025)
- [Logical Structure as Knowledge: Enhancing LLM Reasoning via Structured Logical Knowledge Density Estimation](https://mlsanity.com/paper/2509.24836/) (2025)
- [Beyond Retrieval: A Multitask Benchmark and Model for Code Search](https://mlsanity.com/paper/2605.04615/) (2026)

## Built on by (most cited first, 1 of 1)

- [ZooClaw-FashionSigLIP2: Distilled Fine-tuning for Robust Fashion Retrieval](https://mlsanity.com/paper/2606.27708/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have been widely adopted for synthetic data generation, significantly reducing annotation costs. However, most existing studies treat synthesis as a set of isolated tasks and overlook a more fundamental question: whether a model can learn to synthesize by accumulating experience from past tasks and transferring it to future ones. In this work, we introduce StreamSynth, a new setting in which synthesis tasks arrive sequentially and experience from historical tasks provides informative signals for future synthesis. To address this setting, we propose SynLearner, a general framework that enables synthesis models to acquire reusable synthesis experience over a task stream. Instead of generating data independently for each task, SynLearner encourages the model to explore diverse synthesis patterns, learn from feedback, and balance sample quality with set-level diversity as tasks evolve. Extensive experiments across multiple benchmarks show that SynLearner effectively leverages experience from earlier tasks to improve synthesis performance on later ones, exhibiting consistent cross-task transferability. These findings provide evidence for the feasibility of StreamSynth and highlight synthetic data generation as an experience-driven process that can benefit from task streams.