# Contextually Enhanced ES-dRNN with Dynamic Attention for Short-Term Load Forecasting

arXiv: https://arxiv.org/abs/2212.09030 · HTML: https://mlsanity.com/paper/2212.09030/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 59 references that are in the map.
- First posted on arXiv 18 Dec 2022.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks](https://mlsanity.com/paper/2005.11650/) (2020)
- [Don't Decay the Learning Rate, Increase the Batch Size](https://mlsanity.com/paper/1711.00489/) (2017)
- [A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction](https://mlsanity.com/paper/1704.02971/) (2017)
- [Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting](https://mlsanity.com/paper/1905.03806/) (2019)
- [Adaptive Graph Convolutional Neural Networks](https://mlsanity.com/paper/1801.03226/) (2018)
- [Deep Factors for Forecasting](https://mlsanity.com/paper/1905.12417/) (2019)
- [Kaggle forecasting competitions: An overlooked learning opportunity](https://mlsanity.com/paper/2009.07701/) (2020)
- [Criteria for Classifying Forecasting Methods](https://mlsanity.com/paper/2212.03523/) (2022)
- [Short-term Load Forecasting with Deep Residual Networks](https://mlsanity.com/paper/1805.11956/) (2018)
- [Multivariate Time Series Forecasting with Latent Graph Inference](https://mlsanity.com/paper/2203.03423/) (2022)
- [ES-dRNN: A Hybrid Exponential Smoothing and Dilated Recurrent Neural Network Model for Short-Term Load Forecasting](https://mlsanity.com/paper/2112.02663/) (2021)
- [Random vector functional link neural network based ensemble deep learning for short-term load forecasting](https://mlsanity.com/paper/2107.14385/) (2021)
- [ES-dRNN with Dynamic Attention for Short-Term Load Forecasting](https://mlsanity.com/paper/2203.00937/) (2022)
- [Pattern-based Long Short-term Memory for Mid-term Electrical Load Forecasting](https://mlsanity.com/paper/2004.11834/) (2020)
- [Radflow: A Recurrent, Aggregated, and Decomposable Model for Networks of Time Series](https://mlsanity.com/paper/2102.07289/) (2021)

## Built on by (most cited first, 5 of 5)

- [Stacking for Probabilistic Short-term Load Forecasting](https://mlsanity.com/paper/2406.10718/) (2024)
- [Forecasting Cryptocurrency Prices using Contextual ES-adRNN with Exogenous Variables](https://mlsanity.com/paper/2504.08947/) (2025)
- [Combining Forecasts using Meta-Learning: A Comparative Study for Complex Seasonality](https://mlsanity.com/paper/2504.08940/) (2025)
- [Probabilistic Multi-Regional Solar Power Forecasting with Any-Quantile Recurrent Neural Networks](https://mlsanity.com/paper/2602.05660/) (2026)
- [MACTS-EM: Multi-Agent Collaborative Time Series Forecasting with Emergent Memory](https://mlsanity.com/paper/2610.02255/) (2026)

## Abstract (arXiv)

In this paper, we propose a new short-term load forecasting (STLF) model based on contextually enhanced hybrid and hierarchical architecture combining exponential smoothing (ES) and a recurrent neural network (RNN). The model is composed of two simultaneously trained tracks: the context track and the main track. The context track introduces additional information to the main track. It is extracted from representative series and dynamically modulated to adjust to the individual series forecasted by the main track. The RNN architecture consists of multiple recurrent layers stacked with hierarchical dilations and equipped with recently proposed attentive dilated recurrent cells. These cells enable the model to capture short-term, long-term and seasonal dependencies across time series as well as to weight dynamically the input information. The model produces both point forecasts and predictive intervals. The experimental part of the work performed on 35 forecasting problems shows that the proposed model outperforms in terms of accuracy its predecessor as well as standard statistical models and state-of-the-art machine learning models.