mlsanityData to 14 Oct 2026

CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Data

arXiv 2508.02879 ↗First posted 4 Aug 2025cs.LG cs.AIcited by 13

On the map: Synthetic data and self-improvement for LLMs → Model collapse from synthetic data

Abstract (arXiv)

Time series foundation models (TSFMs) have recently gained significant attention due to their strong zero-shot capabilities and widespread real-world applications. Such models typically require a computationally costly pre-training on large-scale, carefully curated collections of real-world sequences. To allow for a sample-efficient pre-training of TSFMs, we propose textscCauKer, a novel algorithm designed to generate diverse, causally coherent synthetic time series with realistic trends, seasonality, and nonlinear interactions. textscCauKer combines Gaussian Process (GP) kernel composition with Structural Causal Models (SCM) to produce data for sample-efficient pre-training of state-of-the-art classification TSFMs having different architectures and following different pre-training approaches. Additionally, our experiments reveal that textscCauKer-generated datasets exhibit clear scaling laws for both dataset size (10K to 10M samples) and model capacity (1M to 783M parameters), unlike real-world datasets, which display irregular scaling behavior. The source code is publicly available at https://github.com/ShifengXIE/CauKer.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Model collapse from synthetic data, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 41 of its 61 references that are in the map. First posted on arXiv 4 Aug 2025.

Builds on

41 of 61 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Models14 Mar

Other references · 40

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  3. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  4. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 879
  5. 2020Quantifying Attention Flow in Transformerscited by 775
  6. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  7. 2020Self-supervised Learning: Generative or Contrastivecited by 694
  8. 2023A decoder-only foundation model for time-series forecastingcited by 692
  9. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  10. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  11. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 607
  12. 2020A Survey on Contrastive Self-supervised Learningcited by 554
28 more
  1. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  2. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
  3. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 471
  4. 2024The Platonic Representation Hypothesiscited by 382
  5. 2018The UEA multivariate time series classification archive, 2018cited by 338
  6. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 287
  7. 2020WeatherBench: A benchmark dataset for data-driven weather forecastingcited by 275
  8. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 269
  9. 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 212
  10. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 208
  11. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 174
  12. 2022PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecastingcited by 172
  13. 2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasterscited by 129
  14. 2025TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learningcited by 120
  15. 2021Graph-Guided Network for Irregularly Sampled Multivariate Time Seriescited by 111
  16. 2024Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Expertscited by 95
  17. 2024UniTS: A Unified Multi-Task Time Series Modelcited by 94
  18. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 88
  19. 2023Time Series as Images: Vision Transformer for Irregularly Sampled Time Seriescited by 79
  20. 2022A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyondcited by 58
  21. 2024Toto: Time Series Optimized Transformer for Observabilitycited by 48
  22. 2024Towards Neural Scaling Laws for Time Series Foundation Modelscited by 33
  23. 2025Mantis: Lightweight Foundation Model for Time Series Classificationcited by 24
  24. 2023NuTime: Numerically Multi-Scaled Embedding for Large-Scale Time-Series Pretrainingcited by 20
  25. 2025TimesBERT: A BERT-Style Foundation Model for Time Series Understandingcited by 10
  26. 2025COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognitioncited by 9
  27. 2025Unlocking Pretrained Vision Transformers for Time Series Classificationcited by 6
  28. 2023From Alexnet to Transformers: Measuring the Non-linearity of Deep Neural Networks with Affine Optimal Transportcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

By area:

  1. 2026TS-ICL: A Flexible Time-Indexed Foundation Model for Time Series via In-Context Learningcited by 4
  2. 2026TSFMAudit: Data Contamination Auditing in Forecasting Time Series Foundation Modelscited by 3
  3. 2026Post-Training in Time Series Foundation Models: A Unifying Frameworkcited by 3
  4. 2026EXAONE Finance 1.0: An Attention-free Time Series Foundation Model for Financial Time Seriescited by 3
  5. 2026ParisKV: Fast and Drift-Robust KV-Cache Retrieval for Long-Context LLMscited by 2
  6. 2026Aionoscope: Debugging Latent-State Accessibility in Time-Series Representationscited by 2
  7. 2026t0: A Time-Series Foundation Model for Forecasting with Contextcited by 2
  8. 2026A Causal DAG Prior for Synthetic Time-Series Classification Datasetscited by 1
  9. 2026Tabby: An Open Pretraining Recipe for Time Series Foundation Modelscited by 1
  10. 2026Concept Drift from a Causal Perspectivecited by 0
  11. 2026EXAONE Demand 1.0: A Time Series Foundation Model for Demand Forecastingcited by 0
  12. 2026Have an LLM Write Your Anomaly Detector: Autonomous Discovery of Compact, Interpretable Detectors for Time Seriescited by 0
1 more
  1. 2026Prior-Fitted and Frozen Scorers for Multimodal Contrastive Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Chronos: Learning the Language of Time Series
  2. 2026Dissecting Chronos: Sparse Autoencoders Reveal Causal Feature Hierarchies in Time Series Foundation Models
  3. 2026Universal Redundancies in Time Series Foundation Models
  4. 2026Distillation of Synthetic Data for Time Series Foundation Models
  5. 2026Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling
  6. 2024Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization
  7. 2026GRAM: Correcting Frozen Time-Series Foundation Models via Graph-Retrieved Amplitude Memory
  8. 2026When, Not How Much: Evaluating Time-Series Foundation Models on Sparse Events