# ES-dRNN with Dynamic Attention for Short-Term Load Forecasting

arXiv: https://arxiv.org/abs/2203.00937 · HTML: https://mlsanity.com/paper/2203.00937/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 27 references that are in the map.
- First posted on arXiv 2 Mar 2022.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks](https://mlsanity.com/paper/2005.11650/) (2020)
- [Don't Decay the Learning Rate, Increase the Batch Size](https://mlsanity.com/paper/1711.00489/) (2017)
- [Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions](https://mlsanity.com/paper/1909.00590/) (2019)
- [Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark](https://mlsanity.com/paper/2008.08004/) (2020)
- [Short-term Load Forecasting with Deep Residual Networks](https://mlsanity.com/paper/1805.11956/) (2018)
- [ES-dRNN: A Hybrid Exponential Smoothing and Dilated Recurrent Neural Network Model for Short-Term Load Forecasting](https://mlsanity.com/paper/2112.02663/) (2021)
- [A Hybrid Residual Dilated LSTM end Exponential Smoothing Model for Mid-Term Electric Load Forecasting](https://mlsanity.com/paper/2004.00508/) (2020)
- [Pattern-based Long Short-term Memory for Mid-term Electrical Load Forecasting](https://mlsanity.com/paper/2004.11834/) (2020)
- [Randomized Neural Networks for Forecasting Time Series with Multiple Seasonality](https://mlsanity.com/paper/2107.01705/) (2021)

## Built on by (most cited first, 6 of 6)

- [Contextually Enhanced ES-dRNN with Dynamic Attention for Short-Term Load Forecasting](https://mlsanity.com/paper/2212.09030/) (2022)
- [STD: A Seasonal-Trend-Dispersion Decomposition of Time Series](https://mlsanity.com/paper/2204.10398/) (2022)
- [Any-Quantile Probabilistic Forecasting of Short-Term Electricity Demand](https://mlsanity.com/paper/2404.17451/) (2024)
- [Forecasting Cryptocurrency Prices using Contextual ES-adRNN with Exogenous Variables](https://mlsanity.com/paper/2504.08947/) (2025)
- [Recurrent Neural Networks for Forecasting Time Series with Multiple Seasonality: A Comparative Study](https://mlsanity.com/paper/2203.09170/) (2022)
- [A multi-source data power load forecasting method using attention mechanism-based parallel cnn-gru](https://mlsanity.com/paper/2409.17889/) (2024)

## Abstract (arXiv)

Short-term load forecasting (STLF) is a challenging problem due to the complex nature of the time series expressing multiple seasonality and varying variance. This paper proposes an extension of a hybrid forecasting model combining exponential smoothing and dilated recurrent neural network (ES-dRNN) with a mechanism for dynamic attention. We propose a new gated recurrent cell -- attentive dilated recurrent cell, which implements an attention mechanism for dynamic weighting of input vector components. The most relevant components are assigned greater weights, which are subsequently dynamically fine-tuned. This attention mechanism helps the model to select input information and, along with other mechanisms implemented in ES-dRNN, such as adaptive time series processing, cross-learning, and multiple dilation, leads to a significant improvement in accuracy when compared to well-established statistical and state-of-the-art machine learning forecasting models. This was confirmed in the extensive experimental study concerning STLF for 35 European countries.