# Local Linear Forests

arXiv: https://arxiv.org/abs/1807.11408 · HTML: https://mlsanity.com/paper/1807.11408/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jul 2018.

## Built on by (most cited first, 19 of 19)

- [A review of predictive uncertainty estimation with machine learning](https://mlsanity.com/paper/2209.08307/) (2022)
- [Fast Interpretable Greedy-Tree Sums](https://mlsanity.com/paper/2201.11931/) (2022)
- [Random Forest Weighted Local Fr'echet Regression with Random Objects](https://mlsanity.com/paper/2202.04912/) (2022)
- [A review of machine learning concepts and methods for addressing challenges in probabilistic hydrological post-processing and forecasting](https://mlsanity.com/paper/2206.08998/) (2022)
- [A cautionary tale on fitting decision trees to data from additive models: generalization lower bounds](https://mlsanity.com/paper/2110.09626/) (2021)
- [Inference with Mondrian Random Forests](https://mlsanity.com/paper/2310.09702/) (2023)
- [Forest Guided Smoothing](https://mlsanity.com/paper/2103.05092/) (2021)
- [Statistical Inference for Gradient Boosting Regression](https://mlsanity.com/paper/2509.23127/) (2025)
- [Discovering Heterogeneous Treatment Effects in Regression Discontinuity Designs](https://mlsanity.com/paper/2106.11640/) (2021)
- [Causal Clustering for Conditional Average Treatment Effects Estimation and Subgroup Discovery](https://mlsanity.com/paper/2509.05775/) (2025)
- [An Interpretable Neural Network for Parameter Inference](https://mlsanity.com/paper/2106.05536/) (2021)
- [Regression with Missing Data, a Comparison Study of TechniquesBased on Random Forests](https://mlsanity.com/paper/2110.09333/) (2021)
- [Modelling hetegeneous treatment effects by quantitle local polynomial decision tree and forest](https://mlsanity.com/paper/2111.15320/) (2021)
- [Weighted-average quantile regression](https://mlsanity.com/paper/2203.03032/) (2022)
- [Generalized Random Forests using Fixed-Point Trees](https://mlsanity.com/paper/2306.11908/) (2023)
- [SMART: A Flexible Approach to Regression using Spline-Based Multivariate Adaptive Regression Trees](https://mlsanity.com/paper/2410.05597/) (2024)
- [Improving Random Forests by Smoothing](https://mlsanity.com/paper/2505.06852/) (2025)
- [High-Dimensional Dynamic Covariance Models with Random Forests](https://mlsanity.com/paper/2505.12444/) (2025)
- [Interpretable AI with Local Distillation](https://mlsanity.com/paper/2608.23538/) (2026)

## Abstract (arXiv)

Random forests are a powerful method for non-parametric regression, but are limited in their ability to fit smooth signals, and can show poor predictive performance in the presence of strong, smooth effects. Taking the perspective of random forests as an adaptive kernel method, we pair the forest kernel with a local linear regression adjustment to better capture smoothness. The resulting procedure, local linear forests, enables us to improve on asymptotic rates of convergence for random forests with smooth signals, and provides substantial gains in accuracy on both real and simulated data. We prove a central limit theorem valid under regularity conditions on the forest and smoothness constraints, and propose a computationally efficient construction for confidence intervals. Moving to a causal inference application, we discuss the merits of local regression adjustments for heterogeneous treatment effect estimation, and give an example on a dataset exploring the effect word choice has on attitudes to the social safety net. Last, we include simulation results on real and generated data.