# Practical Bayesian Optimization for Model Fitting with Bayesian Adaptive Direct Search

arXiv: https://arxiv.org/abs/1705.04405 · HTML: https://mlsanity.com/paper/1705.04405/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2017.

## Built on by (most cited first, 11 of 11)

- [A Bayesian Optimization Framework for Finding Local Optima in Expensive Multi-Modal Functions](https://mlsanity.com/paper/2210.06635/) (2022)
- [High-dimensional Bayesian Optimization via Covariance Matrix Adaptation Strategy](https://mlsanity.com/paper/2402.03104/) (2024)
- [Bayesian Optimization through Gaussian Cox Process Models for Spatio-temporal Data](https://mlsanity.com/paper/2401.14544/) (2024)
- [PyBADS: Fast and robust black-box optimization in Python](https://mlsanity.com/paper/2306.15576/) (2023)
- [LABCAT: Locally adaptive Bayesian optimization using principal-component-aligned trust regions](https://mlsanity.com/paper/2311.11328/) (2023)
- [Normalizing Flow Regression for Bayesian Inference with Offline Likelihood Evaluations](https://mlsanity.com/paper/2504.11554/) (2025)
- [CobBO: Coordinate Backoff Bayesian Optimization with Two-Stage Kernels](https://mlsanity.com/paper/2101.05147/) (2021)
- [PMBO: Enhancing Black-Box Optimization through Multivariate Polynomial Surrogates](https://mlsanity.com/paper/2403.07485/) (2024)
- [Counterfactual Credit Guided Bayesian Optimization](https://mlsanity.com/paper/2510.04676/) (2025)
- [Surrogate-based Optimization via Clustering for Box-Constrained Problems](https://mlsanity.com/paper/2601.07442/) (2026)
- [LAGO: A Local-Global Optimization Framework Combining Trust Region Methods and Bayesian Optimization](https://mlsanity.com/paper/2603.02970/) (2026)

## Abstract (arXiv)

Computational models in fields such as computational neuroscience are often evaluated via stochastic simulation or numerical approximation. Fitting these models implies a difficult optimization problem over complex, possibly noisy parameter landscapes. Bayesian optimization (BO) has been successfully applied to solving expensive black-box problems in engineering and machine learning. Here we explore whether BO can be applied as a general tool for model fitting. First, we present a novel hybrid BO algorithm, Bayesian adaptive direct search (BADS), that achieves competitive performance with an affordable computational overhead for the running time of typical models. We then perform an extensive benchmark of BADS vs. many common and state-of-the-art nonconvex, derivative-free optimizers, on a set of model-fitting problems with real data and models from six studies in behavioral, cognitive, and computational neuroscience. With default settings, BADS consistently finds comparable or better solutions than other methods, including `vanilla' BO, showing great promise for advanced BO techniques, and BADS in particular, as a general model-fitting tool.