# ScoreGrad: Multivariate Probabilistic Time Series Forecasting with Continuous Energy-based Generative Models

arXiv: https://arxiv.org/abs/2106.10121 · HTML: https://mlsanity.com/paper/2106.10121/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 51 references that are in the map.
- First posted on arXiv 18 Jun 2021.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Sliced Score Matching: A Scalable Approach to Density and Score Estimation](https://mlsanity.com/paper/1905.07088/) (2019)
- [Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2101.12072/) (2021)
- [How to Train Your Energy-Based Models](https://mlsanity.com/paper/2101.03288/) (2021)
- [A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction](https://mlsanity.com/paper/1704.02971/) (2017)
- [A Multi-Horizon Quantile Recurrent Forecaster](https://mlsanity.com/paper/1711.11053/) (2017)
- [High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes](https://mlsanity.com/paper/1910.03002/) (2019)
- [Multivariate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows](https://mlsanity.com/paper/2002.06103/) (2020)
- [Temporal Pattern Attention for Multivariate Time Series Forecasting](https://mlsanity.com/paper/1809.04206/) (2018)
- [A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning](https://mlsanity.com/paper/1710.05741/) (2017)
- [Residual Energy-Based Models for Text Generation](https://mlsanity.com/paper/2004.11714/) (2020)
- [Deep Factors for Forecasting](https://mlsanity.com/paper/1905.12417/) (2019)
- [Dilated Recurrent Neural Networks](https://mlsanity.com/paper/1710.02224/) (2017)
- [The Randomized Midpoint Method for Log-Concave Sampling](https://mlsanity.com/paper/1909.05503/) (2019)
- [GluonTS: Probabilistic Time Series Models in Python](https://mlsanity.com/paper/1906.05264/) (2019)

## Built on by (most cited first, 30 of 43)

- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion](https://mlsanity.com/paper/2407.01392/) (2024)
- [A Survey on Generative Diffusion Model](https://mlsanity.com/paper/2209.02646/) (2022)
- [Foundation Models for Time Series Analysis: A Tutorial and Survey](https://mlsanity.com/paper/2403.14735/) (2024)
- [Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement](https://mlsanity.com/paper/2301.03028/) (2023)
- [Diffusion Models for Time Series Applications: A Survey](https://mlsanity.com/paper/2305.00624/) (2023)
- [Functional Flow Matching](https://mlsanity.com/paper/2305.17209/) (2023)
- [A Survey on Diffusion Models for Time Series and Spatio-Temporal Data](https://mlsanity.com/paper/2404.18886/) (2024)
- [Attacks and Defenses for Generative Diffusion Models: A Comprehensive Survey](https://mlsanity.com/paper/2408.03400/) (2024)
- [The Rise of Diffusion Models in Time-Series Forecasting](https://mlsanity.com/paper/2401.03006/) (2024)
- [Regular Time-series Generation using SGM](https://mlsanity.com/paper/2301.08518/) (2023)
- [Diffusion Models in Bioinformatics: A New Wave of Deep Learning Revolution in Action](https://mlsanity.com/paper/2302.10907/) (2023)
- [Diffusion-SS3D: Diffusion Model for Semi-supervised 3D Object Detection](https://mlsanity.com/paper/2312.02966/) (2023)
- [Elucidating the Design Choice of Probability Paths in Flow Matching for Forecasting](https://mlsanity.com/paper/2410.03229/) (2024)
- [ANT: Adaptive Noise Schedule for Time Series Diffusion Models](https://mlsanity.com/paper/2410.14488/) (2024)
- [DiffsFormer: A Diffusion Transformer on Stock Factor Augmentation](https://mlsanity.com/paper/2402.06656/) (2024)
- [Vision-Enhanced Time Series Forecasting via Latent Diffusion Models](https://mlsanity.com/paper/2502.14887/) (2025)
- [A Comprehensive Survey on Generative Diffusion Models for Structured Data](https://mlsanity.com/paper/2306.04139/) (2023)
- [Navigating Tabular Data Synthesis Research: Understanding User Needs and Tool Capabilities](https://mlsanity.com/paper/2405.20959/) (2024)
- [Constrained Posterior Sampling: Time Series Generation with Hard Constraints](https://mlsanity.com/paper/2410.12652/) (2024)
- [A Financial Time Series Denoiser Based on Diffusion Model](https://mlsanity.com/paper/2409.02138/) (2024)
- [Dynamic data generation and dynamic portfolio selection: an application of a score-based diffusion model](https://mlsanity.com/paper/2507.09916/) (2025)
- [A Temporally Disentangled Contrastive Diffusion Model for Spatiotemporal Imputation](https://mlsanity.com/paper/2402.11558/) (2024)
- [Alternators For Sequence Modeling](https://mlsanity.com/paper/2405.11848/) (2024)
- [This Probably Looks Exactly Like That: An Invertible Prototypical Network](https://mlsanity.com/paper/2407.12200/) (2024)
- [A Dual-Purpose Framework for Backdoor Defense and Backdoor Amplification in Diffusion Models](https://mlsanity.com/paper/2502.19047/) (2025)
- [TimeWak: Temporal Chained-Hashing Watermark for Time Series Data](https://mlsanity.com/paper/2506.06407/) (2025)
- [A multiscale and multicriteria Generative Adversarial Network to synthesize 1-dimensional turbulent fields](https://mlsanity.com/paper/2307.16580/) (2023)
- [TFDPM: Attack detection for cyber-physical systems with diffusion probabilistic models](https://mlsanity.com/paper/2112.10774/) (2021)
- [Accelerated Sequential Flow Matching: A Bayesian Filtering Perspective](https://mlsanity.com/paper/2602.05319/) (2026)

## Abstract (arXiv)

Multivariate time series prediction has attracted a lot of attention because of its wide applications such as intelligence transportation, AIOps. Generative models have achieved impressive results in time series modeling because they can model data distribution and take noise into consideration. However, many existing works can not be widely used because of the constraints of functional form of generative models or the sensitivity to hyperparameters. In this paper, we propose ScoreGrad, a multivariate probabilistic time series forecasting framework based on continuous energy-based generative models. ScoreGrad is composed of time series feature extraction module and conditional stochastic differential equation based score matching module. The prediction can be achieved by iteratively solving reverse-time SDE. To the best of our knowledge, ScoreGrad is the first continuous energy based generative model used for time series forecasting. Furthermore, ScoreGrad achieves state-of-the-art results on six real-world datasets. The impact of hyperparameters and sampler types on the performance are also explored. Code is available at https://github.com/yantijin/ScoreGradPred.