# Balancing Computational Efficiency and Forecast Error in Machine Learning-based Time-Series Forecasting: Insights from Live Experiments on Meteorological Nowcasting

arXiv: https://arxiv.org/abs/2309.15207 · HTML: https://mlsanity.com/paper/2309.15207/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 47 references that are in the map.
- First posted on arXiv 26 Sep 2023.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [GraphCast: Learning skillful medium-range global weather forecasting](https://mlsanity.com/paper/2212.12794/) (2022)
- [Forecasting Global Weather with Graph Neural Networks](https://mlsanity.com/paper/2202.07575/) (2022)
- [From Concept Drift to Model Degradation: An Overview on Performance-Aware Drift Detectors](https://mlsanity.com/paper/2203.11070/) (2022)
- [W-MAE: Pre-trained weather model with masked autoencoder for multi-variable weather forecasting](https://mlsanity.com/paper/2304.08754/) (2023)
- [Efficient Machine Learning for Big Data: A Review](https://mlsanity.com/paper/1503.05296/) (2015)
- [Tropical and Extratropical Cyclone Detection Using Deep Learning](https://mlsanity.com/paper/2005.09056/) (2020)
- [Distributed Deep Learning for Precipitation Nowcasting](https://mlsanity.com/paper/1908.10964/) (2019)
- [A Machine Learning Tutorial for Operational Meteorology, Part I: Traditional Machine Learning](https://mlsanity.com/paper/2204.07492/) (2022)
- [Automatic Environmental Sound Recognition: Performance versus Computational Cost](https://mlsanity.com/paper/1607.04589/) (2016)

## Abstract (arXiv)

Machine learning for time-series forecasting remains a key area of research. Despite successful application of many machine learning techniques, relating computational efficiency to forecast error remains an under-explored domain. This paper addresses this topic through a series of real-time experiments to quantify the relationship between computational cost and forecast error using meteorological nowcasting as an example use-case. We employ a variety of popular regression techniques (XGBoost, FC-MLP, Transformer, and LSTM) for multi-horizon, short-term forecasting of three variables (temperature, wind speed, and cloud cover) for multiple locations. During a 5-day live experiment, 4000 data sources were streamed for training and inferencing 144 models per hour. These models were parameterized to explore forecast error for two computational cost minimization methods: a novel auto-adaptive data reduction technique (Variance Horizon) and a performance-based concept drift-detection mechanism. Forecast error of all model variations were benchmarked in real-time against a state-of-the-art numerical weather prediction model. Performance was assessed using classical and novel evaluation metrics. Results indicate that using the Variance Horizon reduced computational usage by more than 50%, while increasing between 0-15% in error. Meanwhile, performance-based retraining reduced computational usage by up to 90% while also improving forecast error by up to 10%. Finally, the combination of both the Variance Horizon and performance-based retraining outperformed other model configurations by up to 99.7% when considering error normalized to computational usage.