# Attention-based Neural Load Forecasting: A Dynamic Feature Selection Approach

arXiv: https://arxiv.org/abs/2108.11763 · HTML: https://mlsanity.com/paper/2108.11763/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 23 references that are in the map.
- First posted on arXiv 25 Aug 2021.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Recent Trends in Deep Learning Based Natural Language Processing](https://mlsanity.com/paper/1708.02709/) (2017)
- [A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction](https://mlsanity.com/paper/1704.02971/) (2017)
- [Short-term Load Forecasting with Deep Residual Networks](https://mlsanity.com/paper/1805.11956/) (2018)
- [Hourly-Similarity Based Solar Forecasting Using Multi-Model Machine Learning Blending](https://mlsanity.com/paper/1803.03623/) (2018)

## Built on by (most cited first, 3 of 3)

- [Privacy-Preserving Load Forecasting via Personalized Model Obfuscation](https://mlsanity.com/paper/2312.00036/) (2023)
- [A Unifying Framework of Attention-based Neural Load Forecasting](https://mlsanity.com/paper/2305.05082/) (2023)
- [SPECTRA: State-Space Exogenous Context and Temporal-Frequency Resolution Architecture for Probabilistic Energy Forecasting](https://mlsanity.com/paper/2607.20587/) (2026)

## Abstract (arXiv)

Encoder-decoder-based recurrent neural network (RNN) has made significant progress in sequence-to-sequence learning tasks such as machine translation and conversational models. Recent works have shown the advantage of this type of network in dealing with various time series forecasting tasks. The present paper focuses on the problem of multi-horizon short-term load forecasting, which plays a key role in the power system's planning and operation. Leveraging the encoder-decoder RNN, we develop an attention model to select the relevant features and similar temporal information adaptively. First, input features are assigned with different weights by a feature selection attention layer, while the updated historical features are encoded by a bi-directional long short-term memory (BiLSTM) layer. Then, a decoder with hierarchical temporal attention enables a similar day selection, which re-evaluates the importance of historical information at each time step. Numerical results tested on the dataset of the global energy forecasting competition 2014 show that our proposed model significantly outperforms some existing forecasting schemes.