# Iterative Random Forests to detect predictive and stable high-order interactions

arXiv: https://arxiv.org/abs/1706.08457 · HTML: https://mlsanity.com/paper/1706.08457/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2017.

## Built on by (most cited first, 30 of 42)

- [Invariant Causal Mechanisms through Distribution Matching](https://mlsanity.com/paper/2206.11646/) (2022)
- [The Rashomon Importance Distribution: Getting RID of Unstable, Single Model-based Variable Importance](https://mlsanity.com/paper/2309.13775/) (2023)
- [SHAFF: Fast and consistent SHApley eFfect estimates via random Forests](https://mlsanity.com/paper/2105.11724/) (2021)
- [Hierarchical Shrinkage: improving the accuracy and interpretability of tree-based methods](https://mlsanity.com/paper/2202.00858/) (2022)
- [Bagging Provides Assumption-free Stability](https://mlsanity.com/paper/2301.12600/) (2023)
- [Fast Interpretable Greedy-Tree Sums](https://mlsanity.com/paper/2201.11931/) (2022)
- [Random Forest Weighted Local Fr'echet Regression with Random Objects](https://mlsanity.com/paper/2202.04912/) (2022)
- [A cautionary tale on fitting decision trees to data from additive models: generalization lower bounds](https://mlsanity.com/paper/2110.09626/) (2021)
- [PCS-UQ: Uncertainty Quantification via the Predictability-Computability-Stability Framework](https://mlsanity.com/paper/2505.08784/) (2025)
- [Interpretable Machine Learning for Discovery: Statistical Challenges & Opportunities](https://mlsanity.com/paper/2308.01475/) (2023)
- [Machine learning in bioprocess development: From promise to practice](https://mlsanity.com/paper/2210.02200/) (2022)
- [PCS Workflow for Veridical Data Science in the Age of AI](https://mlsanity.com/paper/2508.00835/) (2025)
- [Sanity Checks for Agentic Data Science](https://mlsanity.com/paper/2604.11003/) (2026)
- [Beyond Importance Scores: Interpreting Tabular ML by Visualizing Feature Semantics](https://mlsanity.com/paper/2111.05898/) (2021)
- [iLOCO: Distribution-Free Inference for Feature Interactions](https://mlsanity.com/paper/2502.06661/) (2025)
- [Local MDI+: Local Feature Importances for Tree-Based Models](https://mlsanity.com/paper/2506.08928/) (2025)
- [Individualized and Global Feature Attributions for Gradient Boosted Trees in the Presence of ℓ2 Regularization](https://mlsanity.com/paper/2211.04409/) (2022)
- [Towards Stable Machine Learning Model Retraining via Slowly Varying Sequences](https://mlsanity.com/paper/2403.19871/) (2024)
- [RGMDT: Return-Gap-Minimizing Decision Tree Extraction in Non-Euclidean Metric Space](https://mlsanity.com/paper/2410.16517/) (2024)
- [Selecting Feature Interactions for Generalized Additive Models by Distilling Foundation Models](https://mlsanity.com/paper/2604.13332/) (2026)
- [Machine Learning for Uncovering Biological Insights in Spatial Transcriptomics Data](https://mlsanity.com/paper/2303.16725/) (2023)
- [Interpreting Deep Forest through Feature Contribution and MDI Feature Importance](https://mlsanity.com/paper/2305.00805/) (2023)
- [DeepROCK: Error-controlled interaction detection in deep neural networks](https://mlsanity.com/paper/2309.15319/) (2023)
- [Statistical Advantages of Oblique Randomized Decision Trees and Forests](https://mlsanity.com/paper/2407.02458/) (2024)
- [TrIM: Transformed Iterative Mondrian Forests for Gradient-based Dimension Reduction and High-Dimensional Regression](https://mlsanity.com/paper/2407.09964/) (2024)
- [Lessons from a human-in-the-loop machine learning approach for identifying vacant, abandoned, and deteriorated properties in Savannah, Georgia](https://mlsanity.com/paper/2407.11138/) (2024)
- [Surrogate Interpretable Graph for Random Decision Forests](https://mlsanity.com/paper/2506.01988/) (2025)
- [Decorrelated feature importance from local sample weighting](https://mlsanity.com/paper/2508.06337/) (2025)
- [Learning High-Order Interactions via Targeted Pattern Search](https://mlsanity.com/paper/2102.12974/) (2021)
- [Bagged Polynomial Regression and Neural Networks](https://mlsanity.com/paper/2205.08609/) (2022)

## Abstract (arXiv)

Genomics has revolutionized biology, enabling the interrogation of whole transcriptomes, genome-wide binding sites for proteins, and many other molecular processes. However, individual genomic assays measure elements that interact in vivo as components of larger molecular machines. Understanding how these high-order interactions drive gene expression presents a substantial statistical challenge. Building on Random Forests (RF), Random Intersection Trees (RITs), and through extensive, biologically inspired simulations, we developed the iterative Random Forest algorithm (iRF). iRF trains a feature-weighted ensemble of decision trees to detect stable, high-order interactions with same order of computational cost as RF. We demonstrate the utility of iRF for high-order interaction discovery in two prediction problems: enhancer activity in the early Drosophila embryo and alternative splicing of primary transcripts in human derived cell lines. In Drosophila, among the 20 pairwise transcription factor interactions iRF identifies as stable (returned in more than half of bootstrap replicates), 80% have been previously reported as physical interactions. Moreover, novel third-order interactions, e.g. between Zelda (Zld), Giant (Gt), and Twist (Twi), suggest high-order relationships that are candidates for follow-up experiments. In human-derived cells, iRF re-discovered a central role of H3K36me3 in chromatin-mediated splicing regulation, and identified novel 5th and 6th order interactions, indicative of multi-valent nucleosomes with specific roles in splicing regulation. By decoupling the order of interactions from the computational cost of identification, iRF opens new avenues of inquiry into the molecular mechanisms underlying genome biology.