# Regression Trees Know Calculus

arXiv: https://arxiv.org/abs/2405.13846 · HTML: https://mlsanity.com/paper/2405.13846/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 48 references that are in the map.
- First posted on arXiv 22 May 2024.

## Builds on (most cited first, 17 of 17)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Machine Learning of Linear Differential Equations using Gaussian Processes](https://mlsanity.com/paper/1701.02440/) (2017)
- [True to the Model or True to the Data?](https://mlsanity.com/paper/2006.16234/) (2020)
- [Unrestricted Permutation forces Extrapolation: Variable Importance Requires at least One More Model, or There Is No Free Variable Importance](https://mlsanity.com/paper/1905.03151/) (2019)
- [Deep Learning of Vortex Induced Vibrations](https://mlsanity.com/paper/1808.08952/) (2018)
- [Explainable AI for Trees: From Local Explanations to Global Understanding](https://mlsanity.com/paper/1905.04610/) (2019)
- [Understanding Random Forests: From Theory to Practice](https://mlsanity.com/paper/1407.7502/) (2014)
- [Using Attribution to Decode Dataset Bias in Neural Network Models for Chemistry](https://mlsanity.com/paper/1811.11310/) (2018)
- [Interpretation of Prediction Models Using the Input Gradient](https://mlsanity.com/paper/1611.07634/) (2016)
- [A Debiased MDI Feature Importance Measure for Random Forests](https://mlsanity.com/paper/1906.10845/) (2019)
- [Sequential Learning of Active Subspaces](https://mlsanity.com/paper/1907.11572/) (2019)
- [Epistemic values in feature importance methods: Lessons from feminist epistemology](https://mlsanity.com/paper/2101.12737/) (2021)
- [Deep active subspaces - a scalable method for high-dimensional uncertainty propagation](https://mlsanity.com/paper/1902.10527/) (2019)
- [Nonparametric Variable Screening with Optimal Decision Stumps](https://mlsanity.com/paper/2011.02683/) (2020)
- [Sensitivity Prewarping for Local Surrogate Modeling](https://mlsanity.com/paper/2101.06296/) (2021)

## Abstract (arXiv)

Regression trees have emerged as a preeminent tool for solving real-world regression problems due to their ability to deal with nonlinearities, interaction effects and sharp discontinuities. In this article, we rather study regression trees applied to well-behaved, differentiable functions, and determine the relationship between node parameters and the local gradient of the function being approximated. We find a simple estimate of the gradient which can be efficiently computed using quantities exposed by popular tree learning libraries. This allows the tools developed in the context of differentiable algorithms, like neural nets and Gaussian processes, to be deployed to tree-based models. To demonstrate this, we study measures of model sensitivity defined in terms of integrals of gradients and demonstrate how to compute them for regression trees using the proposed gradient estimates. Quantitative and qualitative numerical experiments reveal the capability of gradients estimated by regression trees to improve predictive analysis, solve tasks in uncertainty quantification, and provide interpretation of model behavior.