# Embedded feature selection in LSTM networks with multi-objective evolutionary ensemble learning for time series forecasting

arXiv: https://arxiv.org/abs/2312.17517 · HTML: https://mlsanity.com/paper/2312.17517/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 56 references that are in the map.
- First posted on arXiv 29 Dec 2023.

## Builds on (most cited first, 9 of 9)

- [An Improved Analysis of Stochastic Gradient Descent with Momentum](https://mlsanity.com/paper/2007.07989/) (2020)
- [EA-LSTM: Evolutionary Attention-based LSTM for Time Series Prediction](https://mlsanity.com/paper/1811.03760/) (2018)
- [AFS: An Attention-based mechanism for Supervised Feature Selection](https://mlsanity.com/paper/1902.11074/) (2019)
- [Deep Feature Selection using a Teacher-Student Network](https://mlsanity.com/paper/1903.07045/) (2019)
- [FocusLearn: Fully-Interpretable, High-Performance Modular Neural Networks for Time Series](https://mlsanity.com/paper/2311.16834/) (2023)
- [E2E-FS: An End-to-End Feature Selection Method for Neural Networks](https://mlsanity.com/paper/2012.07671/) (2020)
- [Feature Selection for Multivariate Time Series via Network Pruning](https://mlsanity.com/paper/2102.06024/) (2021)
- [Embedded methods for feature selection in neural networks](https://mlsanity.com/paper/2010.05834/) (2020)
- [An Embedded Feature Selection Framework for Control](https://mlsanity.com/paper/2206.11064/) (2022)

## Built on by (most cited first, 1 of 1)

- [Unveiling the Power of Sparse Neural Networks for Feature Selection](https://mlsanity.com/paper/2408.04583/) (2024)

## Abstract (arXiv)

Time series forecasting plays a crucial role in diverse fields, necessitating the development of robust models that can effectively handle complex temporal patterns. In this article, we present a novel feature selection method embedded in Long Short-Term Memory networks, leveraging a multi-objective evolutionary algorithm. Our approach optimizes the weights and biases of the LSTM in a partitioned manner, with each objective function of the evolutionary algorithm targeting the root mean square error in a specific data partition. The set of non-dominated forecast models identified by the algorithm is then utilized to construct a meta-model through stacking-based ensemble learning. Furthermore, our proposed method provides an avenue for attribute importance determination, as the frequency of selection for each attribute in the set of non-dominated forecasting models reflects their significance. This attribute importance insight adds an interpretable dimension to the forecasting process. Experimental evaluations on air quality time series data from Italy and southeast Spain demonstrate that our method substantially improves the generalization ability of conventional LSTMs, effectively reducing overfitting. Comparative analyses against state-of-the-art CancelOut and EAR-FS methods highlight the superior performance of our approach.