# Generalizing Gain Penalization for Feature Selection in Tree-based Models

arXiv: https://arxiv.org/abs/2006.07515 · HTML: https://mlsanity.com/paper/2006.07515/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2020.

## Built on by (most cited first, 1 of 1)

- [What is to be gained by ensemble models in analysis of spectroscopic data?](https://mlsanity.com/paper/2404.02184/) (2024)

## Abstract (arXiv)

We develop a new approach for feature selection via gain penalization in tree-based models. First, we show that previous methods do not perform sufficient regularization and often exhibit sub-optimal out-of-sample performance, especially when correlated features are present. Instead, we develop a new gain penalization idea that exhibits a general local-global regularization for tree-based models. The new method allows for more flexibility in the choice of feature-specific importance weights. We validate our method on both simulated and real data and implement itas an extension of the popular R package ranger.