# Multi-scale Attention Flow for Probabilistic Time Series Forecasting

arXiv: https://arxiv.org/abs/2205.07493 · HTML: https://mlsanity.com/paper/2205.07493/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 71 references that are in the map.
- First posted on arXiv 16 May 2022.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting](https://mlsanity.com/paper/2103.07719/) (2021)
- [A note on the evaluation of generative models](https://mlsanity.com/paper/1511.01844/) (2015)
- [Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019](https://mlsanity.com/paper/1911.13288/) (2019)
- [Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design](https://mlsanity.com/paper/1902.00275/) (2019)
- [Structured Inference Networks for Nonlinear State Space Models](https://mlsanity.com/paper/1609.09869/) (2016)
- [Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting](https://mlsanity.com/paper/1905.03806/) (2019)
- [Multivariate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows](https://mlsanity.com/paper/2002.06103/) (2020)
- [PixelSNAIL: An Improved Autoregressive Generative Model](https://mlsanity.com/paper/1712.09763/) (2017)
- [Temporal Pattern Attention for Multivariate Time Series Forecasting](https://mlsanity.com/paper/1809.04206/) (2018)
- [CTNet: Context-based Tandem Network for Semantic Segmentation](https://mlsanity.com/paper/2104.09805/) (2021)
- [Particle Filter Recurrent Neural Networks](https://mlsanity.com/paper/1905.12885/) (2019)

## Built on by (most cited first, 19 of 19)

- [Detecting Multivariate Time Series Anomalies with Zero Known Label](https://mlsanity.com/paper/2208.02108/) (2022)
- [Label-Free Multivariate Time Series Anomaly Detection](https://mlsanity.com/paper/2312.11549/) (2023)
- [W-Transformers : A Wavelet-based Transformer Framework for Univariate Time Series Forecasting](https://mlsanity.com/paper/2209.03945/) (2022)
- [Hierarchical Skeleton Meta-Prototype Contrastive Learning with Hard Skeleton Mining for Unsupervised Person Re-Identification](https://mlsanity.com/paper/2307.12917/) (2023)
- [Multimodal Conditioned Diffusive Time Series Forecasting](https://mlsanity.com/paper/2504.19669/) (2025)
- [K2VAE: A Koopman-Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2505.23017/) (2025)
- [Modeling Temporal Dependencies within the Target for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2406.04777/) (2024)
- [Physics-Informed Real NVP for Satellite Power System Fault Detection](https://mlsanity.com/paper/2405.17339/) (2024)
- [ContextFlow++: Generalist-Specialist Flow-based Generative Models with Mixed-Variable Context Encoding](https://mlsanity.com/paper/2406.00578/) (2024)
- [UmambaTSF: A U-shaped Multi-Scale Long-Term Time Series Forecasting Method Using Mamba](https://mlsanity.com/paper/2410.11278/) (2024)
- [MSDformer: Multi-scale Discrete Transformer For Time Series Generation](https://mlsanity.com/paper/2505.14202/) (2025)
- [Beyond All-to-All: Causal-Aligned Transformer with Dynamic Structure Learning for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2505.16308/) (2025)
- [Knowledge-based Deep Learning for Modeling Chaotic Systems](https://mlsanity.com/paper/2209.04259/) (2022)
- [Probabilistic Learning of Multivariate Time Series with Temporal Irregularity](https://mlsanity.com/paper/2306.09147/) (2023)
- [MGCP: A Multi-Grained Correlation based Prediction Network for Multivariate Time Series](https://mlsanity.com/paper/2405.19661/) (2024)
- [Interpretable deep convolutional model for nonlinear multivariate time series in complex systems](https://mlsanity.com/paper/2501.04339/) (2025)
- [GluMind: Multimodal Parallel Attention and Knowledge Retention for Robust Cross-Population Blood Glucose Forecasting](https://mlsanity.com/paper/2509.18457/) (2025)
- [Beyond Static Uncertainty: Modeling Temporal Uncertainty Dynamics for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2603.24254/) (2026)
- [ProbRes: Volatility Learning for Probabilistic Time-Series Forecasting](https://mlsanity.com/paper/2606.02117/) (2026)

## Abstract (arXiv)

The probability prediction of multivariate time series is a notoriously challenging but practical task. On the one hand, the challenge is how to effectively capture the cross-series correlations between interacting time series, to achieve accurate distribution modeling. On the other hand, we should consider how to capture the contextual information within time series more accurately to model multivariate temporal dynamics of time series. In this work, we proposed a novel non-autoregressive deep learning model, called Multi-scale Attention Normalizing Flow(MANF), where we integrate multi-scale attention and relative position information and the multivariate data distribution is represented by the conditioned normalizing flow. Additionally, compared with autoregressive modeling methods, our model avoids the influence of cumulative error and does not increase the time complexity. Extensive experiments demonstrate that our model achieves state-of-the-art performance on many popular multivariate datasets.