mlsanityData to 7 Oct 2026

TimeDiT: General-purpose Diffusion Transformers for Time Series Foundation Model

arXiv 2409.02322 ↗First posted 3 Sep 2024cs.LG cs.AIcited by 33

On the map: Diffusion models → Flow matching, time series and science

Abstract (arXiv)

Foundation models, particularly Large Language Models (LLMs), have revolutionized text and video processing, yet time series data presents distinct challenges for such approaches due to domain-specific features such as missing values, multi-resolution characteristics, etc. Furthermore, the de-facto autoregressive transformers tend to learn deterministic temporal dependencies within pre-trained data while overlooking inherent uncertainties and lacking integration of physical constraints. In this paper, we introduce TimeDiT, a diffusion transformer model that synergistically combines transformer-based temporal dependency learning with diffusion-based probabilistic sampling. TimeDiT employs a unified masking mechanism to harmonize the training and inference process across diverse tasks while introducing a theoretically grounded, finetuning-free model editing strategy that enables flexible integration of external knowledge during sampling. Acknowledging the challenges of unifying multiple downstream tasks under a single model, our systematic evaluation demonstrates TimeDiT's effectiveness both in fundamental tasks, i.e., forecasting and imputation, through zero-shot/fine-tuning; and in domain tasks, i.e., multi-resolution forecasting, anomaly detection, and data generation, establishing it as a textitproto-foundation model that bridges the gap between general-purpose and domain-specific models.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in Flow matching, time series and science, one of 14 directions in Diffusion models. Builds on 61 of its 101 references that are in the map. First posted on arXiv 3 Sep 2024.

Builds on

61 of 101 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024A Survey on Diffusion Models for Time Series and Spatio-Temporal Data29 Apr
  2. 2024Diffusion-TS: Interpretable Diffusion for General Time Series Generation4 Mar
  3. 2023Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecasting21 Jul
  4. 2021CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation7 Jul
  5. 2021Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting28 Jan

Other references · 56

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2022Scalable Diffusion Models with Transformerscited by 5,931
  7. 2017Neural Discrete Representation Learningcited by 4,650
  8. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  9. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  10. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  11. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  12. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
38 more
  1. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  2. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
  3. 2024Chronos: Learning the Language of Time Seriescited by 712
  4. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  5. 2023A decoder-only foundation model for time-series forecastingcited by 650
  6. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  7. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 603
  8. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  9. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  10. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  11. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 463
  12. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
  13. 2018Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholdingcited by 407
  14. 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 396
  15. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 387
  16. 2021Spectral Temporal Graph Neural Network for Multivariate Time-series Forecastingcited by 298
  17. 2021Monash Time Series Forecasting Archivecited by 256
  18. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
  19. 2018Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applicationscited by 223
  20. 2022TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Datacited by 211
  21. 2019Towards Physics-informed Deep Learning for Turbulent Flow Predictioncited by 203
  22. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 170
  23. 2020Multivariate Time-series Anomaly Detection via Graph Attention Networkcited by 165
  24. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 165
  25. 2019Time-Series Anomaly Detection Service at Microsoftcited by 163
  26. 2023TimeGPT-1cited by 149
  27. 2019High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processescited by 147
  28. 2021TimeVAE: A Variational Auto-Encoder for Multivariate Time Series Generationcited by 139
  29. 2023SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modelingcited by 112
  30. 2023VDT: General-purpose Video Diffusion Transformers via Mask Modelingcited by 102
  31. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  32. 2023Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglementcited by 89
  33. 2024Empowering Time Series Analysis with Large Language Models: A Surveycited by 60
  34. 2020Identifying Coordinated Accounts on Social Media through Hidden Influence and Group Behaviourscited by 46
  35. 2024BitsFusion: 1.99 bits Weight Quantization of Diffusion Modelcited by 37
  36. 2022Counterfactual Neural Temporal Point Process for Estimating Causal Influence of Misinformation on Social Mediacited by 20
  37. 2023Estimating Treatment Effects from Irregular Time Series Observations with Hidden Confounderscited by 17
  38. 2024DisDet: Exploring Detectability of Backdoor Attack on Diffusion Modelscited by 17

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 16; 2026: 16.

33 papers.

Sort:

By area:

  1. 2025Foundation Models for Spatio-Temporal Data Science: A Tutorial and Surveycited by 23
  2. 2025A Survey of Reasoning and Agentic Systems in Time Series with Large Language Modelscited by 21
  3. 2025T2S: High-resolution Time Series Generation with Text-to-Series Diffusion Modelscited by 19
  4. 2026TemporalBench: A Benchmark for Evaluating LLM-Based Agents on Contextual and Event-Informed Time Series Taskscited by 15
  5. 2025Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approachcited by 14
  6. 2025A Survey on Diffusion Models for Anomaly Detectioncited by 13
  7. 2025Anomaly Detection and Generation with Diffusion Models: A Surveycited by 8
  8. 2025Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Modelscited by 5
  9. 2025Foundation Models for Demand Forecasting via Dual-Strategy Ensemblingcited by 5
  10. 2026Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Surveycited by 3
  11. 2024Channel-aware Contrastive Conditional Diffusion for Multivariate Probabilistic Time Series Forecastingcited by 2
  12. 2025Are Time-Series Foundation Models Deployment-Ready? A Systematic Study of Adversarial Robustness Across Domainscited by 2
21 more
  1. 2025Forecasting Time Series with LLMs via Patch-Based Prompting and Decompositioncited by 2
  2. 2026TimeMar: Multi-Scale Autoregressive Modeling for Unconditional Time Series Generationcited by 2
  3. 2025How to Unlock Time Series Editing? Diffusion-Driven Approach with Multi-Grained Controlcited by 1
  4. 2025SDSC:A Structure-Aware Metric for Semantic Signal Representation Learningcited by 1
  5. 2026TF-CoDiT: Conditional Time Series Synthesis with Diffusion Transformers for Treasury Futurescited by 1
  6. 2026DiTS: Multimodal Diffusion Transformers Are Time Series Forecasterscited by 1
  7. 2026TRACE: A Temporal Conditional Estimation for Multimodal Time Series Foundation Modelscited by 1
  8. 2025Instruction-based Time Series Editingcited by 0
  9. 2025Diffusion-Based Scenario Tree Generation for Multivariate Time Series Prediction and Multistage Stochastic Optimizationcited by 0
  10. 2025Diffusion Transformers for Imputation: Statistical Efficiency and Uncertainty Quantificationcited by 0
  11. 2025Foundation Model Forecasts: Form and Functioncited by 0
  12. 2026DP-aware AdaLN-Zero: Taming Conditioning-Induced Heavy-Tailed Gradients in Differentially Private Diffusioncited by 0
  13. 2026Conditional Imputation for Within-Modality Missingness in Multi-Modal Federated Learningcited by 0
  14. 2026Speaking Numbers to LLMs: Multi-Wavelet Number Embeddings for Time Series Forecastingcited by 0
  15. 2026When Denoising Hurts: Rethinking the Terminal Step of Diffusion Time Series Forecasters -- Extended Versioncited by 0
  16. 2026Coarse-to-Fine Multi-Resolution Diffusion Models for Trajectory Generation in Urban Systemscited by 0
  17. 2026LiveHouse-TS: An Open-world Living Benchmark for Time Series Foundation Modelscited by 0
  18. 2026WaveTLM: Reliable Time-Series Language Modeling through Task Compilationcited by 0
  19. 2026KiT: A Foundation Model for Financial Time-Series Forecasting using DiffusionTransformerscited by 0
  20. 2026Energy Time-Series Imputation with Differentially Private Diffusion Models via Clipping-Aware Objective Conditioningcited by 0
  21. 2026ProtoFlow: Prototype-Guided Flow Matching for Multivariate Time Series Forecastingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards a Unified Generative Model for Scarce Time Series with Domain Experts
  2. 2024TimeDART: A Diffusion Autoregressive Transformer for Self-Supervised Time Series Representation
  3. 2025TIMED: Adversarial and Autoregressive Refinement of Diffusion-Based Time Series Generation
  4. 2026Enhancing few-shot time series forecasting with LLM-guided diffusion
  5. 2024TimeLDM: Latent Diffusion Model for Unconditional Time Series Generation
  6. 2026Distribution-Aware Diffusion-LLM for Robust Ultra-Long-Term Time Series Forecasting
  7. 2023Diffusion Models for Time Series Applications: A Survey
  8. 2025TimeDP: Learning to Generate Multi-Domain Time Series with Domain Prompts