# Distributional Split Criteria for Random Forests: Extensions, Shrinkage, and the Robustness of Mean Splitting

arXiv: https://arxiv.org/abs/2607.23721 · HTML: https://mlsanity.com/paper/2607.23721/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 4 references that are in the map.
- First posted on arXiv 26 Jul 2026.

## Builds on (most cited first, 2 of 2)

- [Distributional Random Forests: Heterogeneity Adjustment and Multivariate Distributional Regression](https://mlsanity.com/paper/2005.14458/) (2020)
- [Computational-Statistical Trade-off in Kernel Two-Sample Testing with Random Fourier Features](https://mlsanity.com/paper/2407.08976/) (2024)

## Abstract (arXiv)

Distributional random forests replace mean-based CART splitting with criteria that compare the full conditional response distribution in candidate children. We implement and systematically study a family of such criteria inside a single honest-forest implementation: isotropic random-Fourier-feature maximum mean discrepancy (MMD), an anisotropic diagonal-bandwidth variant, an adaptive per-split frequency-selection variant, and a non-kernel sliced-Wasserstein criterion, together with post-hoc kernel-mean shrinkage of the forest weights. Using paired-seed comparisons across synthetic quantile mechanisms, real univariate benchmarks, a California-housing subsample curve, and multivariate synthetic and real responses, we characterize where each extension pays. Three findings recur. First, among distributional criteria ordinary isotropic MMD is already close to best in class: the anisotropic, adaptive-frequency, and sliced-Wasserstein extensions, and post-hoc shrinkage, do not systematically improve on it. Second, on scalar tabular regression mean-based CART splitting remains the robust default and wins many cells. Third, multivariate responses are the regime where distributional splitting clearly earns its keep, most sharply on a pure-dependence copula where the energy score separates the criteria even though marginal CRPS does not. The evidence supports a simple allocation story: distributional splitting helps only when non-location structure is both present and estimable; otherwise it dilutes split-selection power away from the mean. All criteria, the honest forest, and the paired-comparison harness are implemented in the open-source textttdrforest library, whose Rust-backed split search makes broad criterion sweeps inexpensive.