# Hierarchical Ensemble-Based Feature Selection for Time Series Forecasting

arXiv: https://arxiv.org/abs/2310.17544 · HTML: https://mlsanity.com/paper/2310.17544/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 36 references that are in the map.
- First posted on arXiv 26 Oct 2023.

## Builds on (most cited first, 7 of 7)

- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [Relief-Based Feature Selection: Introduction and Review](https://mlsanity.com/paper/1711.08421/) (2017)
- [Koopman Neural Forecaster for Time Series with Temporal Distribution Shifts](https://mlsanity.com/paper/2210.03675/) (2022)
- [Incremental Permutation Feature Importance (iPFI): Towards Online Explanations on Data Streams](https://mlsanity.com/paper/2209.01939/) (2022)
- [A User-Guided Bayesian Framework for Ensemble Feature Selection in Life Science Applications (UBayFS)](https://mlsanity.com/paper/2104.14787/) (2021)
- [Improving LSTM Neural Networks for Better Short-Term Wind Power Predictions](https://mlsanity.com/paper/1907.00489/) (2019)
- [ReliefE: Feature Ranking in High-dimensional Spaces via Manifold Embeddings](https://mlsanity.com/paper/2101.09577/) (2021)

## Abstract (arXiv)

We introduce a novel ensemble approach for feature selection based on hierarchical stacking for non-stationarity and/or a limited number of samples with a large number of features. Our approach exploits the co-dependency between features using a hierarchical structure. Initially, a machine learning model is trained using a subset of features, and then the output of the model is updated using other algorithms in a hierarchical manner with the remaining features to minimize the target loss. This hierarchical structure allows for flexible depth and feature selection. By exploiting feature co-dependency hierarchically, our proposed approach overcomes the limitations of traditional feature selection methods and feature importance scores. The effectiveness of the approach is demonstrated on synthetic and well-known real-life datasets, providing significant scalable and stable performance improvements compared to the traditional methods and the state-of-the-art approaches. We also provide the source code of our approach to facilitate further research and replicability of our results.