# Evaluating State of the Art, Forecasting Ensembles- and Meta-learning Strategies for Model Fusion

arXiv: https://arxiv.org/abs/2203.03279 · HTML: https://mlsanity.com/paper/2203.03279/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 72 references that are in the map.
- First posted on arXiv 7 Mar 2022.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Dilated Recurrent Neural Networks](https://mlsanity.com/paper/1710.02224/) (2017)
- [An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation](https://mlsanity.com/paper/2008.09586/) (2020)
- [The Schulze Method of Voting](https://mlsanity.com/paper/1804.02973/) (2018)
- [Multivariate Anomaly Detection based on Prediction Intervals Constructed using Deep Learning](https://mlsanity.com/paper/2110.03393/) (2021)
- [A Statistics and Deep Learning Hybrid Method for Multivariate Time Series Forecasting and Mortality Modeling](https://mlsanity.com/paper/2112.08618/) (2021)
- [Time series model selection with a meta-learning approach; evidence from a pool of forecasting algorithms](https://mlsanity.com/paper/1908.08489/) (2019)
- [Feature-weighted Stacking for Nonseasonal Time Series Forecasts: A Case Study of the COVID-19 Epidemic Curves](https://mlsanity.com/paper/2108.08723/) (2021)
- [Surrogate Parameters Optimization for Data and Model Fusion of COVID-19 Time-series Data](https://mlsanity.com/paper/2109.04207/) (2021)

## Built on by (most cited first, 3 of 3)

- [Volatility forecasting using Deep Learning and sentiment analysis](https://mlsanity.com/paper/2210.12464/) (2022)
- [FRANS: Automatic Feature Extraction for Time Series Forecasting](https://mlsanity.com/paper/2209.07018/) (2022)
- [Late Meta-learning Fusion Using Representation Learning for Time Series Forecasting](https://mlsanity.com/paper/2303.11000/) (2023)

## Abstract (arXiv)

Techniques of hybridisation and ensemble learning are popular model fusion techniques for improving the predictive power of forecasting methods. With limited research that instigates combining these two promising approaches, this paper focuses on the utility of the Exponential-Smoothing-Recurrent Neural Network (ES-RNN) in the pool of base models for different ensembles. We compare against some state of the art ensembling techniques and arithmetic model averaging as a benchmark. We experiment with the M4 forecasting data set of 100,000 time-series, and the results show that the Feature-based Forecast Model Averaging (FFORMA), on average, is the best technique for late data fusion with the ES-RNN. However, considering the M4's Daily subset of data, stacking was the only successful ensemble at dealing with the case where all base model performances are similar. Our experimental results indicate that we attain state of the art forecasting results compared to N-BEATS as a benchmark. We conclude that model averaging is a more robust ensemble than model selection and stacking strategies. Further, the results show that gradient boosting is superior for implementing ensemble learning strategies.