# TimeDiT: General-purpose Diffusion Transformers for Time Series Foundation Model

arXiv: https://arxiv.org/abs/2409.02322 · HTML: https://mlsanity.com/paper/2409.02322/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in Flow matching, time series and science, one of 14 directions in Diffusion models.
- Builds on 61 of its 101 references that are in the map.
- First posted on arXiv 3 Sep 2024.

## Placed in

- Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/)

## Builds on (most cited first, 30 of 61)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation](https://mlsanity.com/paper/2107.03502/) (2021)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding](https://mlsanity.com/paper/1802.04431/) (2018)
- [Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy](https://mlsanity.com/paper/2110.02642/) (2021)
- [Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2101.12072/) (2021)
- [Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting](https://mlsanity.com/paper/2103.07719/) (2021)

## Built on by (most cited first, 30 of 33)

- [Foundation Models for Spatio-Temporal Data Science: A Tutorial and Survey](https://mlsanity.com/paper/2503.13502/) (2025)
- [A Survey of Reasoning and Agentic Systems in Time Series with Large Language Models](https://mlsanity.com/paper/2509.11575/) (2025)
- [T2S: High-resolution Time Series Generation with Text-to-Series Diffusion Models](https://mlsanity.com/paper/2505.02417/) (2025)
- [TemporalBench: A Benchmark for Evaluating LLM-Based Agents on Contextual and Event-Informed Time Series Tasks](https://mlsanity.com/paper/2602.13272/) (2026)
- [Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach](https://mlsanity.com/paper/2505.20446/) (2025)
- [A Survey on Diffusion Models for Anomaly Detection](https://mlsanity.com/paper/2501.11430/) (2025)
- [Anomaly Detection and Generation with Diffusion Models: A Survey](https://mlsanity.com/paper/2506.09368/) (2025)
- [Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models](https://mlsanity.com/paper/2505.10930/) (2025)
- [Foundation Models for Demand Forecasting via Dual-Strategy Ensembling](https://mlsanity.com/paper/2507.22053/) (2025)
- [Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Survey](https://mlsanity.com/paper/2601.03181/) (2026)
- [Channel-aware Contrastive Conditional Diffusion for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2410.02168/) (2024)
- [Are Time-Series Foundation Models Deployment-Ready? A Systematic Study of Adversarial Robustness Across Domains](https://mlsanity.com/paper/2505.19397/) (2025)
- [Forecasting Time Series with LLMs via Patch-Based Prompting and Decomposition](https://mlsanity.com/paper/2506.12953/) (2025)
- [TimeMar: Multi-Scale Autoregressive Modeling for Unconditional Time Series Generation](https://mlsanity.com/paper/2601.11184/) (2026)
- [TRACE: A Temporal Conditional Estimation for Multimodal Time Series Foundation Models](https://mlsanity.com/paper/2606.06285/) (2026)
- [How to Unlock Time Series Editing? Diffusion-Driven Approach with Multi-Grained Control](https://mlsanity.com/paper/2506.05276/) (2025)
- [SDSC:A Structure-Aware Metric for Semantic Signal Representation Learning](https://mlsanity.com/paper/2507.14516/) (2025)
- [Instruction-based Time Series Editing](https://mlsanity.com/paper/2508.01504/) (2025)
- [TF-CoDiT: Conditional Time Series Synthesis with Diffusion Transformers for Treasury Futures](https://mlsanity.com/paper/2601.11880/) (2026)
- [DiTS: Multimodal Diffusion Transformers Are Time Series Forecasters](https://mlsanity.com/paper/2602.06597/) (2026)
- [DP-aware AdaLN-Zero: Taming Conditioning-Induced Heavy-Tailed Gradients in Differentially Private Diffusion](https://mlsanity.com/paper/2602.22610/) (2026)
- [Diffusion-Based Scenario Tree Generation for Multivariate Time Series Prediction and Multistage Stochastic Optimization](https://mlsanity.com/paper/2509.14832/) (2025)
- [Diffusion Transformers for Imputation: Statistical Efficiency and Uncertainty Quantification](https://mlsanity.com/paper/2510.02216/) (2025)
- [Foundation Model Forecasts: Form and Function](https://mlsanity.com/paper/2510.19345/) (2025)
- [Conditional Imputation for Within-Modality Missingness in Multi-Modal Federated Learning](https://mlsanity.com/paper/2604.23112/) (2026)
- [Speaking Numbers to LLMs: Multi-Wavelet Number Embeddings for Time Series Forecasting](https://mlsanity.com/paper/2606.26487/) (2026)
- [When Denoising Hurts: Rethinking the Terminal Step of Diffusion Time Series Forecasters -- Extended Version](https://mlsanity.com/paper/2608.14067/) (2026)
- [Coarse-to-Fine Multi-Resolution Diffusion Models for Trajectory Generation in Urban Systems](https://mlsanity.com/paper/2608.14570/) (2026)
- [LiveHouse-TS: An Open-world Living Benchmark for Time Series Foundation Models](https://mlsanity.com/paper/2608.17299/) (2026)
- [WaveTLM: Reliable Time-Series Language Modeling through Task Compilation](https://mlsanity.com/paper/2609.18812/) (2026)

## Abstract (arXiv)

Foundation models, particularly Large Language Models (LLMs), have revolutionized text and video processing, yet time series data presents distinct challenges for such approaches due to domain-specific features such as missing values, multi-resolution characteristics, etc. Furthermore, the de-facto autoregressive transformers tend to learn deterministic temporal dependencies within pre-trained data while overlooking inherent uncertainties and lacking integration of physical constraints. In this paper, we introduce TimeDiT, a diffusion transformer model that synergistically combines transformer-based temporal dependency learning with diffusion-based probabilistic sampling. TimeDiT employs a unified masking mechanism to harmonize the training and inference process across diverse tasks while introducing a theoretically grounded, finetuning-free model editing strategy that enables flexible integration of external knowledge during sampling. Acknowledging the challenges of unifying multiple downstream tasks under a single model, our systematic evaluation demonstrates TimeDiT's effectiveness both in fundamental tasks, i.e., forecasting and imputation, through zero-shot/fine-tuning; and in domain tasks, i.e., multi-resolution forecasting, anomaly detection, and data generation, establishing it as a textitproto-foundation model that bridges the gap between general-purpose and domain-specific models.