# Causal Feature Selection via Transfer Entropy

arXiv: https://arxiv.org/abs/2310.11059 · HTML: https://mlsanity.com/paper/2310.11059/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 45 references that are in the map.
- First posted on arXiv 17 Oct 2023.

## Builds on (most cited first, 15 of 15)

- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Detecting causal associations in large nonlinear time series datasets](https://mlsanity.com/paper/1702.07007/) (2017)
- [Causality for Machine Learning](https://mlsanity.com/paper/1911.10500/) (2019)
- [Discovering contemporaneous and lagged causal relations in autocorrelated nonlinear time series datasets](https://mlsanity.com/paper/2003.03685/) (2020)
- [Invariance, Causality and Robustness](https://mlsanity.com/paper/1812.08233/) (2018)
- [High-recall causal discovery for autocorrelated time series with latent confounders](https://mlsanity.com/paper/2007.01884/) (2020)
- [Conditional independence testing based on a nearest-neighbor estimator of conditional mutual information](https://mlsanity.com/paper/1709.01447/) (2017)
- [Feature Selection: A Data Perspective](https://mlsanity.com/paper/1601.07996/) (2016)
- [Causal Inference for Time series Analysis: Problems, Methods and Evaluation](https://mlsanity.com/paper/2102.05829/) (2021)
- [Estimating Mutual Information for Discrete-Continuous Mixtures](https://mlsanity.com/paper/1709.06212/) (2017)
- [Feature Selection via Mutual Information: New Theoretical Insights](https://mlsanity.com/paper/1907.07384/) (2019)
- [Variable-lag Granger Causality and Transfer Entropy for Time Series Analysis](https://mlsanity.com/paper/2002.00208/) (2020)
- [Exponential Concentration of a Density Functional Estimator](https://mlsanity.com/paper/1603.08584/) (2016)
- [Generalized Exponential Concentration Inequality for R'enyi Divergence Estimation](https://mlsanity.com/paper/1603.08589/) (2016)
- [Estimating Conditional Transfer Entropy in Time Series using Mutual Information and Non-linear Prediction](https://mlsanity.com/paper/2011.13655/) (2020)

## Built on by (most cited first, 3 of 3)

- [TERC: A Transfer Entropy Redundancy Criterion for State Variable Selection in Reinforcement Learning](https://mlsanity.com/paper/2401.11512/) (2024)
- [TREET: TRansfer Entropy Estimation via Transformers](https://mlsanity.com/paper/2402.06919/) (2024)
- [MotifDisco: Motif Causal Discovery For Time Series Motifs](https://mlsanity.com/paper/2409.15219/) (2024)

## Abstract (arXiv)

Machine learning algorithms are designed to capture complex relationships between features. In this context, the high dimensionality of data often results in poor model performance, with the risk of overfitting. Feature selection, the process of selecting a subset of relevant and non-redundant features, is, therefore, an essential step to mitigate these issues. However, classical feature selection approaches do not inspect the causal relationship between selected features and target, which can lead to misleading results in real-world applications. Causal discovery, instead, aims to identify causal relationships between features with observational data. In this paper, we propose a novel methodology at the intersection between feature selection and causal discovery, focusing on time series. We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures and leverages transfer entropy to estimate the causal flow of information from the features to the target in time series. Our approach enables the selection of features not only in terms of mere model performance but also captures the causal information flow. In this context, we provide theoretical guarantees on the regression and classification errors for both the exact and the finite-sample cases. Finally, we present numerical validations on synthetic and real-world regression problems, showing results competitive w.r.t. the considered baselines.