# DART: Dropouts meet Multiple Additive Regression Trees

arXiv: https://arxiv.org/abs/1505.01866 · HTML: https://mlsanity.com/paper/1505.01866/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2015.

## Built on by (most cited first, 15 of 15)

- [Towards Understanding Fairness and its Composition in Ensemble Machine Learning](https://mlsanity.com/paper/2212.04593/) (2022)
- [TREE-G: Decision Trees Contesting Graph Neural Networks](https://mlsanity.com/paper/2207.02760/) (2022)
- [IDP-PGFE: An Interpretable Disruption Predictor based on Physics-Guided Feature Extraction](https://mlsanity.com/paper/2208.13197/) (2022)
- [Statistical Inference for Gradient Boosting Regression](https://mlsanity.com/paper/2509.23127/) (2025)
- [RelAgent: LLM Agents as Data Scientists for Relational Learning](https://mlsanity.com/paper/2605.07840/) (2026)
- [Comparing Machine Learning and Interpolation Methods for Loop-Level Calculations](https://mlsanity.com/paper/2111.14788/) (2021)
- [Interpretable Boosted Decision Tree Analysis for the Majorana Demonstrator](https://mlsanity.com/paper/2207.10710/) (2022)
- [Online learning techniques for prediction of temporal tabular datasets with regime changes](https://mlsanity.com/paper/2301.00790/) (2022)
- [Cross-tokamak Disruption Prediction based on Physics-Guided Feature Extraction and domain adaptation](https://mlsanity.com/paper/2309.05361/) (2023)
- [From Point to probabilistic gradient boosting for claim frequency and severity prediction](https://mlsanity.com/paper/2412.14916/) (2024)
- [Continuous Attribution of Episodical Outcomes for More Efficient and Targeted Online Measurement](https://mlsanity.com/paper/2210.16373/) (2022)
- [Morphological Classification of Extragalactic Radio Sources Using Gradient Boosting Methods](https://mlsanity.com/paper/2304.12729/) (2023)
- [AKEM: Aligning Knowledge Base to Queries with Ensemble Model for Entity Recognition and Linking](https://mlsanity.com/paper/2309.06175/) (2023)
- [Noninvasive Estimation of Mean Pulmonary Artery Pressure Using MRI, Computer Models, and Machine Learning](https://mlsanity.com/paper/2312.14221/) (2023)
- [Geo-Semantic-Parsing: AI-powered geoparsing by traversing semantic knowledge graphs](https://mlsanity.com/paper/2503.01386/) (2025)

## Abstract (arXiv)

Multiple Additive Regression Trees (MART), an ensemble model of boosted regression trees, is known to deliver high prediction accuracy for diverse tasks, and it is widely used in practice. However, it suffers an issue which we call over-specialization, wherein trees added at later iterations tend to impact the prediction of only a few instances, and make negligible contribution towards the remaining instances. This negatively affects the performance of the model on unseen data, and also makes the model over-sensitive to the contributions of the few, initially added tress. We show that the commonly used tool to address this issue, that of shrinkage, alleviates the problem only to a certain extent and the fundamental issue of over-specialization still remains. In this work, we explore a different approach to address the problem that of employing dropouts, a tool that has been recently proposed in the context of learning deep neural networks. We propose a novel way of employing dropouts in MART, resulting in the DART algorithm. We evaluate DART on ranking, regression and classification tasks, using large scale, publicly available datasets, and show that DART outperforms MART in each of the tasks, with a significant margin. We also show that DART overcomes the issue of over-specialization to a considerable extent.