# Conditional Inference Trees and Forests for Feature Selection

arXiv: https://arxiv.org/abs/2607.01417 · HTML: https://mlsanity.com/paper/2607.01417/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 36 references that are in the map.
- First posted on arXiv 1 Jul 2026.

## Builds on (most cited first, 4 of 4)

- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [The Randomized Dependence Coefficient](https://mlsanity.com/paper/1304.7717/) (2013)

## Abstract (arXiv)

Conditional inference trees (CIT) and conditional inference forests (CIF) reduce split-selection bias by testing features before choosing split thresholds, but repeated permutation tests and threshold searches can make these methods computationally expensive. We study CIT and CIF as top-k feature-ranking methods for downstream prediction using real-data benchmarks, runtime ablations, and synthetic feature-recovery experiments. At a fixed node, if the features and permutation budget do not depend on the node responses, Bonferroni-corrected +1 Monte Carlo permutation p-values control nodewise rejection under the complete permutation null. CIF ranks 4th among 17 classification methods on 22 datasets and 3rd among 18 regression methods on 8 datasets. With Bonferroni correction held fixed, the CIF runtime ablations indicate that adaptive stopping and the number of thresholds searched have the largest measured effect on runtime: turning off adaptive stopping and using exact threshold search increase fitting time by 4.0--8.4× and 1.9--10.8×, respectively, while downstream score changes are at most 0.011. Sparse high-p simulations indicate that forest feature sampling can leave informative features out of many split decisions. Overall, the results support CIF as a top-k feature-ranking method in the evaluated downstream prediction benchmarks.