# Finding Influential Training Samples for Gradient Boosted Decision Trees

arXiv: https://arxiv.org/abs/1802.06640 · HTML: https://mlsanity.com/paper/1802.06640/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2018.

## Built on by (most cited first, 15 of 15)

- [Training Data Influence Analysis and Estimation: A Survey](https://mlsanity.com/paper/2212.04612/) (2022)
- [The Shapley Value in Machine Learning](https://mlsanity.com/paper/2202.05594/) (2022)
- [HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks](https://mlsanity.com/paper/2102.02515/) (2021)
- [Data Debugging with Shapley Importance over End-to-End Machine Learning Pipelines](https://mlsanity.com/paper/2204.11131/) (2022)
- [Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Trees](https://mlsanity.com/paper/2205.00359/) (2022)
- [Differentially Private Shapley Values for Data Evaluation](https://mlsanity.com/paper/2206.00511/) (2022)
- [Addressing Budget Allocation and Revenue Allocation in Data Market Environments Using an Adaptive Sampling Algorithm](https://mlsanity.com/paper/2306.02543/) (2023)
- [AXIL: Exact Instance Attribution for Gradient Boosting](https://mlsanity.com/paper/2301.01864/) (2023)
- [Fast-DataShapley: Neural Modeling for Training Data Valuation](https://mlsanity.com/paper/2506.05281/) (2025)
- [Thresholding Data Shapley for Data Cleansing Using Multi-Armed Bandits](https://mlsanity.com/paper/2402.08209/) (2024)
- [Unsupervised Synthetic Image Attribution: Alignment and Disentanglement](https://mlsanity.com/paper/2601.22663/) (2026)
- [Towards the Use of Saliency Maps for Explaining Low-Quality Electrocardiograms to End Users](https://mlsanity.com/paper/2207.02726/) (2022)
- [Accelerated Shapley Value Approximation for Data Evaluation](https://mlsanity.com/paper/2311.05346/) (2023)
- [LiveVal: Time-aware Data Valuation via Adaptive Reference Points](https://mlsanity.com/paper/2502.10489/) (2025)
- [MUC-FL: Block-Wise Marginal Utility Contribution for Communication-Efficient Federated Learning](https://mlsanity.com/paper/2609.10545/) (2026)

## Abstract (arXiv)

We address the problem of finding influential training samples for a particular case of tree ensemble-based models, e.g., Random Forest (RF) or Gradient Boosted Decision Trees (GBDT). A natural way of formalizing this problem is studying how the model's predictions change upon leave-one-out retraining, leaving out each individual training sample. Recent work has shown that, for parametric models, this analysis can be conducted in a computationally efficient way. We propose several ways of extending this framework to non-parametric GBDT ensembles under the assumption that tree structures remain fixed. Furthermore, we introduce a general scheme of obtaining further approximations to our method that balance the trade-off between performance and computational complexity. We evaluate our approaches on various experimental setups and use-case scenarios and demonstrate both the quality of our approach to finding influential training samples in comparison to the baselines and its computational efficiency.