# Global and Preference-based Optimization with Mixed Variables using Piecewise Affine Surrogates

arXiv: https://arxiv.org/abs/2302.04686 · HTML: https://mlsanity.com/paper/2302.04686/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 67 references that are in the map.
- First posted on arXiv 9 Feb 2023.

## Builds on (most cited first, 12 of 12)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [NAS-Bench-101: Towards Reproducible Neural Architecture Search](https://mlsanity.com/paper/1902.09635/) (2019)
- [A Literature Survey of Benchmark Functions For Global Optimization Problems](https://mlsanity.com/paper/1308.4008/) (2013)
- [Tabular Benchmarks for Joint Architecture and Hyperparameter Optimization](https://mlsanity.com/paper/1905.04970/) (2019)
- [Bayesian Optimisation over Multiple Continuous and Categorical Inputs](https://mlsanity.com/paper/1906.08878/) (2019)
- [Mixed-Integer Convex Nonlinear Optimization with Gradient-Boosted Trees Embedded](https://mlsanity.com/paper/1803.00952/) (2018)
- [Global optimization via inverse distance weighting and radial basis functions](https://mlsanity.com/paper/1906.06498/) (2019)
- [Constrained Discrete Black-Box Optimization using Mixed-Integer Programming](https://mlsanity.com/paper/2110.09569/) (2021)
- [A general mathematical framework for constrained mixed-variable blackbox optimization problems with meta and categorical variables](https://mlsanity.com/paper/2204.00886/) (2022)
- [Efficient Calibration of Embedded MPC](https://mlsanity.com/paper/1911.13021/) (2019)
- [DIRECTGO: A new DIRECT-type MATLAB toolbox for derivative-free global optimization](https://mlsanity.com/paper/2107.02205/) (2021)
- [NOMAD version 4: Nonlinear optimization with the MADS algorithm](https://mlsanity.com/paper/2104.11627/) (2021)

## Abstract (arXiv)

Optimization problems involving mixed variables (i.e., variables of numerical and categorical nature) can be challenging to solve, especially in the presence of mixed-variable constraints. Moreover, when the objective function is the result of a complicated simulation or experiment, it may be expensive-to-evaluate. This paper proposes a novel surrogate-based global optimization algorithm to solve linearly constrained mixed-variable problems up to medium size (around 100 variables after encoding). The proposed approach is based on constructing a piecewise affine surrogate of the objective function over feasible samples. We assume the objective function is black-box and expensive-to-evaluate, while the linear constraints are quantifiable, unrelaxable, a priori known, and are cheap to evaluate. We introduce two types of exploration functions to efficiently search the feasible domain via mixed-integer linear programming solvers. We also provide a preference-based version of the algorithm designed for situations where only pairwise comparisons between samples can be acquired, while the underlying objective function to minimize remains unquantified. The two algorithms are evaluated on several unconstrained and constrained mixed-variable benchmark problems. The results show that, within a small number of required experiments/simulations, the proposed algorithms can often achieve better or comparable results than other existing methods.