# A better method to enforce monotonic constraints in regression and classification trees

arXiv: https://arxiv.org/abs/2011.00986 · HTML: https://mlsanity.com/paper/2011.00986/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2020.

## Built on by (most cited first, 3 of 3)

- [Expressive Monotonic Neural Networks](https://mlsanity.com/paper/2307.07512/) (2023)
- [Robust and Provably Monotonic Networks](https://mlsanity.com/paper/2112.00038/) (2021)
- [Shape-constrained Symbolic Regression with NSGA-III](https://mlsanity.com/paper/2209.13851/) (2022)

## Abstract (arXiv)

In this report we present two new ways of enforcing monotone constraints in regression and classification trees. One yields better results than the current LightGBM, and has a similar computation time. The other one yields even better results, but is much slower than the current LightGBM. We also propose a heuristic that takes into account that greedily splitting a tree by choosing a monotone split with respect to its immediate gain is far from optimal. Then, we compare the results with the current implementation of the constraints in the LightGBM library, using the well known Adult public dataset. Throughout the report, we mostly focus on the implementation of our methods that we made for the LightGBM library, even though they are general and could be implemented in any regression or classification tree. The best method we propose (a smarter way to split the tree coupled to a penalization of monotone splits) consistently beats the current implementation of LightGBM. With small or average trees, the loss reduction can be as high as 1% in the early stages of training and decreases to around 0.1% at the loss peak for the Adult dataset. The results would be even better with larger trees. In our experiments, we didn't do a lot of tuning of the regularization parameters, and we wouldn't be surprised to see that increasing the performance of our methods on test sets.