Model Selection for Time Series Forecasting: Empirical Analysis of Different Estimators
Not on the map. Why?
Abstract (arXiv)
Evaluating predictive models is a crucial task in predictive analytics. This process is especially challenging with time series data where the observations show temporal dependencies. Several studies have analysed how different performance estimation methods compare with each other for approximating the true loss incurred by a given forecasting model. However, these studies do not address how the estimators behave for model selection: the ability to select the best solution among a set of alternatives. We address this issue and compare a set of estimation methods for model selection in time series forecasting tasks. We attempt to answer two main questions: (i) how often is the best possible model selected by the estimators; and (ii) what is the performance loss when it does not. We empirically found that the accuracy of the estimators for selecting the best solution is low, and the overall forecasting performance loss associated with the model selection process ranges from 1.2% to 2.3%. We also discovered that some factors, such as the sample size, are important in the relative performance of the estimators.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 11 of its 58 references that are in the map. First posted on arXiv 1 Apr 2021.
Builds on
11 of 58 references.
References · 11
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
- 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,013
- 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 761
- 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 753
- 2015ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and Rcited by 172
- 2015Should we really use post-hoc tests based on mean-ranks?cited by 116
- 2019Evaluating time series forecasting models: An empirical study on performance estimation methodscited by 53
- 2020Improving the Accuracy of Global Forecasting Models using Time Series Data Augmentationcited by 31
- 2020Global Models for Time Series Forecasting: A Simulation Studycited by 21
- 2019Machine Learning vs Statistical Methods for Time Series Forecasting: Size Matterscited by 10
- 2018How to evaluate sentiment classifiers for Twitter time-ordered data?cited by 5
Built on by
Citing papers per year: 2024: 2; 2025: 1.
3 papers.