# Masked Multi-Step Multivariate Time Series Forecasting with Future Information

arXiv: https://arxiv.org/abs/2209.14413 · HTML: https://mlsanity.com/paper/2209.14413/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 40 references that are in the map.
- First posted on arXiv 28 Sep 2022.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [Time Series Forecasting With Deep Learning: A Survey](https://mlsanity.com/paper/2004.13408/) (2020)
- [Tabular Transformers for Modeling Multivariate Time Series](https://mlsanity.com/paper/2011.01843/) (2020)
- [Gated Transformer Networks for Multivariate Time Series Classification](https://mlsanity.com/paper/2103.14438/) (2021)
- [Multi-Step-Ahead Time Series Prediction using Multiple-Output Support Vector Regression](https://mlsanity.com/paper/1401.2504/) (2014)
- [Recurrent Neural Filters: Learning Independent Bayesian Filtering Steps for Time Series Prediction](https://mlsanity.com/paper/1901.08096/) (2019)
- [Time Series Generation with Masked Autoencoder](https://mlsanity.com/paper/2201.07006/) (2022)
- [MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Series](https://mlsanity.com/paper/2205.02100/) (2022)
- [Time-series Transformer Generative Adversarial Networks](https://mlsanity.com/paper/2205.11164/) (2022)
- [ForecastNet: A Time-Variant Deep Feed-Forward Neural Network Architecture for Multi-Step-Ahead Time-Series Forecasting](https://mlsanity.com/paper/2002.04155/) (2020)
- [Flight Demand Forecasting with Transformers](https://mlsanity.com/paper/2111.04471/) (2021)
- [Deep Learning for Flight Demand Forecasting](https://mlsanity.com/paper/2011.04476/) (2020)

## Built on by (most cited first, 2 of 2)

- [Masked Multi-Step Probabilistic Forecasting for Short-to-Mid-Term Electricity Demand](https://mlsanity.com/paper/2302.06818/) (2023)
- [One Masked Model is All You Need for Sensor Fault Detection, Isolation and Accommodation](https://mlsanity.com/paper/2403.16153/) (2024)

## Abstract (arXiv)

In this paper, we introduce Masked Multi-Step Multivariate Forecasting (MMMF), a novel and general self-supervised learning framework for time series forecasting with known future information. In many real-world forecasting scenarios, some future information is known, e.g., the weather information when making a short-to-mid-term electricity demand forecast, or the oil price forecasts when making an airplane departure forecast. Existing machine learning forecasting frameworks can be categorized into (1) sample-based approaches where each forecast is made independently, and (2) time series regression approaches where the future information is not fully incorporated. To overcome the limitations of existing approaches, we propose MMMF, a framework to train any neural network model capable of generating a sequence of outputs, that combines both the temporal information from the past and the known information about the future to make better predictions. Experiments are performed on two real-world datasets for (1) mid-term electricity demand forecasting, and (2) two-month ahead flight departures forecasting. They show that the proposed MMMF framework outperforms not only sample-based methods but also existing time series forecasting models with the exact same base models. Furthermore, once a neural network model is trained with MMMF, its inference speed is similar to that of the same model trained with traditional regression formulations, thus making MMMF a better alternative to existing regression-trained time series forecasting models if there is some available future information.