# Invariant Random Forest: Tree-Based Model Solution for OOD Generalization

arXiv: https://arxiv.org/abs/2312.04273 · HTML: https://mlsanity.com/paper/2312.04273/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 7 Dec 2023.

## Builds on (most cited first, 13 of 13)

- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [The Risks of Invariant Risk Minimization](https://mlsanity.com/paper/2010.05761/) (2020)
- [Deep Stable Learning for Out-Of-Distribution Generalization](https://mlsanity.com/paper/2104.07876/) (2021)
- [Stable Learning via Sample Reweighting](https://mlsanity.com/paper/1911.12580/) (2019)
- [Stable Prediction with Model Misspecification and Agnostic Distribution Shift](https://mlsanity.com/paper/2001.11713/) (2020)
- [Stable Prediction across Unknown Environments](https://mlsanity.com/paper/1806.06270/) (2018)
- [Stable Adversarial Learning under Distributional Shifts](https://mlsanity.com/paper/2006.04414/) (2020)
- [Stable Learning via Sparse Variable Independence](https://mlsanity.com/paper/2212.00992/) (2022)
- [Quantitatively Measuring and Contrastively Exploring Heterogeneity for Domain Generalization](https://mlsanity.com/paper/2305.15889/) (2023)
- [Decorr: Environment Partitioning for Invariant Learning and OOD Generalization](https://mlsanity.com/paper/2211.10054/) (2022)

## Built on by (most cited first, 1 of 1)

- [Decorrelated feature importance from local sample weighting](https://mlsanity.com/paper/2508.06337/) (2025)

## Abstract (arXiv)

Out-Of-Distribution (OOD) generalization is an essential topic in machine learning. However, recent research is only focusing on the corresponding methods for neural networks. This paper introduces a novel and effective solution for OOD generalization of decision tree models, named Invariant Decision Tree (IDT). IDT enforces a penalty term with regard to the unstable/varying behavior of a split across different environments during the growth of the tree. Its ensemble version, the Invariant Random Forest (IRF), is constructed. Our proposed method is motivated by a theoretical result under mild conditions, and validated by numerical tests with both synthetic and real datasets. The superior performance compared to non-OOD tree models implies that considering OOD generalization for tree models is absolutely necessary and should be given more attention.