# Mixed-Integer Convex Nonlinear Optimization with Gradient-Boosted Trees Embedded

arXiv: https://arxiv.org/abs/1803.00952 · HTML: https://mlsanity.com/paper/1803.00952/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2018.

## Built on by (most cited first, 16 of 16)

- [OMLT: Optimization & Machine Learning Toolkit](https://mlsanity.com/paper/2202.02414/) (2022)
- [Maximizing information from chemical engineering data sets: Applications to machine learning](https://mlsanity.com/paper/2201.10035/) (2022)
- [Tree ensemble kernels for Bayesian optimization with known constraints over mixed-feature spaces](https://mlsanity.com/paper/2207.00879/) (2022)
- [Multi-Objective Constrained Optimization for Energy Applications via Tree Ensembles](https://mlsanity.com/paper/2111.03140/) (2021)
- [PySCIPOpt-ML: Embedding Trained Machine Learning Models into Mixed-Integer Programs](https://mlsanity.com/paper/2312.08074/) (2023)
- [Machine learning for industrial sensing and control: A survey and practical perspective](https://mlsanity.com/paper/2401.13836/) (2024)
- [Global Optimization: A Machine Learning Approach](https://mlsanity.com/paper/2311.01742/) (2023)
- [BARK: A Fully Bayesian Tree Kernel for Black-box Optimization](https://mlsanity.com/paper/2503.05574/) (2025)
- [Global optimization of graph acquisition functions for neural architecture search](https://mlsanity.com/paper/2505.23640/) (2025)
- [Optimizing over an ensemble of neural networks](https://mlsanity.com/paper/2112.07007/) (2021)
- [Constraint Learning to Define Trust Regions in Predictive-Model Embedded Optimization](https://mlsanity.com/paper/2201.04429/) (2022)
- [Multi-Objective Linear Ensembles for Robust and Sparse Training of Few-Bit Neural Networks](https://mlsanity.com/paper/2212.03659/) (2022)
- [Tight Mixed-Integer Optimization Formulations for Prescriptive Trees](https://mlsanity.com/paper/2302.14744/) (2023)
- [Conformal Mixed-Integer Constraint Learning with Feasibility Guarantees](https://mlsanity.com/paper/2506.03531/) (2025)
- [Global and Preference-based Optimization with Mixed Variables using Piecewise Affine Surrogates](https://mlsanity.com/paper/2302.04686/) (2023)
- [Surrogate-based Optimization via Clustering for Box-Constrained Problems](https://mlsanity.com/paper/2601.07442/) (2026)

## Abstract (arXiv)

Decision trees usefully represent sparse, high dimensional and noisy data. Having learned a function from this data, we may want to thereafter integrate the function into a larger decision-making problem, e.g., for picking the best chemical process catalyst. We study a large-scale, industrially-relevant mixed-integer nonlinear nonconvex optimization problem involving both gradient-boosted trees and penalty functions mitigating risk. This mixed-integer optimization problem with convex penalty terms broadly applies to optimizing pre-trained regression tree models. Decision makers may wish to optimize discrete models to repurpose legacy predictive models, or they may wish to optimize a discrete model that particularly well-represents a data set. We develop several heuristic methods to find feasible solutions, and an exact, branch-and-bound algorithm leveraging structural properties of the gradient-boosted trees and penalty functions. We computationally test our methods on concrete mixture design instance and a chemical catalysis industrial instance.