# A Statistics and Deep Learning Hybrid Method for Multivariate Time Series Forecasting and Mortality Modeling

arXiv: https://arxiv.org/abs/2112.08618 · HTML: https://mlsanity.com/paper/2112.08618/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 56 references that are in the map.
- First posted on arXiv 16 Dec 2021.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches](https://mlsanity.com/paper/1803.04386/) (2018)
- [TensorFlow Distributions](https://mlsanity.com/paper/1711.10604/) (2017)
- [Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with NBEATSx](https://mlsanity.com/paper/2104.05522/) (2021)
- [What Teachers Should Know about the Bootstrap: Resampling in the Undergraduate Statistics Curriculum](https://mlsanity.com/paper/1411.5279/) (2014)
- [Deep learning via LSTM models for COVID-19 infection forecasting in India](https://mlsanity.com/paper/2101.11881/) (2021)
- [Multivariate Anomaly Detection based on Prediction Intervals Constructed using Deep Learning](https://mlsanity.com/paper/2110.03393/) (2021)
- [A spatio-temporal LSTM model to forecast across multiple temporal and spatial scales](https://mlsanity.com/paper/2108.11875/) (2021)
- [Fast ES-RNN: A GPU Implementation of the ES-RNN Algorithm](https://mlsanity.com/paper/1907.03329/) (2019)

## Built on by (most cited first, 6 of 6)

- [A Survey on Explainable Anomaly Detection](https://mlsanity.com/paper/2210.06959/) (2022)
- [Evaluating State of the Art, Forecasting Ensembles- and Meta-learning Strategies for Model Fusion](https://mlsanity.com/paper/2203.03279/) (2022)
- [Statistics and Deep Learning-based Hybrid Model for Interpretable Anomaly Detection](https://mlsanity.com/paper/2202.12720/) (2022)
- [Strict baselines for Covid-19 forecasting and ML perspective for USA and Russia](https://mlsanity.com/paper/2207.07689/) (2022)
- [Towards a methodology for addressing missingness in datasets, with an application to demographic health datasets](https://mlsanity.com/paper/2211.02856/) (2022)
- [Late Meta-learning Fusion Using Representation Learning for Time Series Forecasting](https://mlsanity.com/paper/2303.11000/) (2023)

## Abstract (arXiv)

Hybrid methods have been shown to outperform pure statistical and pure deep learning methods at forecasting tasks and quantifying the associated uncertainty with those forecasts (prediction intervals). One example is Exponential Smoothing Recurrent Neural Network (ES-RNN), a hybrid between a statistical forecasting model and a recurrent neural network variant. ES-RNN achieves a 9.4% improvement in absolute error in the Makridakis-4 Forecasting Competition. This improvement and similar outperformance from other hybrid models have primarily been demonstrated only on univariate datasets. Difficulties with applying hybrid forecast methods to multivariate data include (i) the high computational cost involved in hyperparameter tuning for models that are not parsimonious, (ii) challenges associated with auto-correlation inherent in the data, as well as (iii) complex dependency (cross-correlation) between the covariates that may be hard to capture. This paper presents Multivariate Exponential Smoothing Long Short Term Memory (MES-LSTM), a generalized multivariate extension to ES-RNN, that overcomes these challenges. MES-LSTM utilizes a vectorized implementation. We test MES-LSTM on several aggregated coronavirus disease of 2019 (COVID-19) morbidity datasets and find our hybrid approach shows consistent, significant improvement over pure statistical and deep learning methods at forecast accuracy and prediction interval construction.