mlsanityData to 14 Oct 2026

Mix, Don't Pick: Why Synthetic Corpus Composition Matters for Time Series Foundation Model Pretraining

arXiv 2606.09912 ↗First posted 6 Jun 2026cs.LG cs.AIcited by 1

On the map: Synthetic data and self-improvement for LLMs → Model collapse from synthetic data

Abstract (arXiv)

Choosing the wrong synthetic generator for time-series foundation model pretraining is costly: under identical training budgets, the best and worst generators produce up to a 2× gap in forecasting error, yet the field has no principled way to make this choice. The problem is compounded by the fact that generator rankings are not stable across architectures: across 11 generator families evaluated on Chronos-T5-Mini and Moirai-Small trained from scratch, we find that which generators are useful depends on the model architecture. Rather than solving the generator selection problem, we sidestep it: a simple equal-weight mixture of all generators matches or beats the best individual generator for both architectures, and composing this mixture with real data yields the strongest pretraining corpora overall. Synthetic pretraining is therefore a corpus composition problem, not a generator selection problem, and composition choices should be validated per model family rather than assumed to transfer.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Model collapse from synthetic data, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 6 of its 24 references that are in the map. First posted on arXiv 6 Jun 2026.

Builds on

6 of 24 references.

References · 6

  1. 2024Chronos: Learning the Language of Time Seriescited by 759
  2. 2023A decoder-only foundation model for time-series forecastingcited by 692
  3. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  4. 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 212
  5. 2019catch22: CAnonical Time-series CHaracteristicscited by 105
  6. 2025TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecastingcited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026IND-Eval: Aligning Evaluation and Simulation for Industrial Time Series Foundation Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Does Synthetic Data Help? Empirical Evidence from Deep Learning Time Series Forecasters
  2. 2025Investigating Compositional Reasoning in Time Series Foundation Models
  3. 2025Characterizing Model Behavior Under Synthetic Data Training: An Empirical Study Across Scales and Mixing Ratios
  4. 2025Conversational Time Series Foundation Models: Towards Explainable and Effective Forecasting
  5. 2025Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls
  6. 2026How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data
  7. 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
  8. 2024Unveiling the Flaws: Exploring Imperfections in Synthetic Data and Mitigation Strategies for Large Language Models