# Practical Bayesian optimization in the presence of outliers

arXiv: https://arxiv.org/abs/1712.04567 · HTML: https://mlsanity.com/paper/1712.04567/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2017.

## Built on by (most cited first, 25 of 25)

- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [Forking Uncertainties: Reliable Prediction and Model Predictive Control with Sequence Models via Conformal Risk Control](https://mlsanity.com/paper/2310.10299/) (2023)
- [Multi-Fidelity Bayesian Optimization with Unreliable Information Sources](https://mlsanity.com/paper/2210.13937/) (2022)
- [Distributionally Robust Bayesian Optimization with varphi-divergences](https://mlsanity.com/paper/2203.02128/) (2022)
- [Robust and Conjugate Gaussian Process Regression](https://mlsanity.com/paper/2311.00463/) (2023)
- [A Robust Phased Elimination Algorithm for Corruption-Tolerant Gaussian Process Bandits](https://mlsanity.com/paper/2202.01850/) (2022)
- [Scalable First-Order Bayesian Optimization via Structured Automatic Differentiation](https://mlsanity.com/paper/2206.08366/) (2022)
- [Robust Bayesian Learning for Reliable Wireless AI: Framework and Applications](https://mlsanity.com/paper/2207.00300/) (2022)
- [Machine-learning-accelerated simulations to enable automatic surface reconstruction](https://mlsanity.com/paper/2305.07251/) (2023)
- [Robust PACm: Training Ensemble Models Under Misspecification and Outliers](https://mlsanity.com/paper/2203.01859/) (2022)
- [Adversarial Attacks on Gaussian Process Bandits](https://mlsanity.com/paper/2110.08449/) (2021)
- [ODBO: Bayesian Optimization with Search Space Prescreening for Directed Protein Evolution](https://mlsanity.com/paper/2205.09548/) (2022)
- [Quantum approximate optimization via learning-based adaptive optimization](https://mlsanity.com/paper/2303.14877/) (2023)
- [The effects of data size on Automated Essay Scoring engines](https://mlsanity.com/paper/2108.13275/) (2021)
- [Learning Ordering in Crystalline Materials with Symmetry-Aware Graph Neural Networks](https://mlsanity.com/paper/2409.13851/) (2024)
- [Robust Satisficing Gaussian Process Bandits Under Adversarial Attacks](https://mlsanity.com/paper/2506.01625/) (2025)
- [Robust Bayesian Optimisation with Unbounded Corruptions](https://mlsanity.com/paper/2511.15315/) (2025)
- [Machine learning for rapid discovery of laminar flow channel wall modifications that enhance heat transfer](https://mlsanity.com/paper/2101.08130/) (2021)
- [One-parameter family of acquisition functions for efficient global optimization](https://mlsanity.com/paper/2104.12363/) (2021)
- [Density Ratio Estimation-based Bayesian Optimization with Semi-Supervised Learning](https://mlsanity.com/paper/2305.15612/) (2023)
- [Beyond Regrets: Geometric Metrics for Bayesian Optimization](https://mlsanity.com/paper/2401.01981/) (2024)
- [Active Learning for Manifold Gaussian Process Regression](https://mlsanity.com/paper/2506.20928/) (2025)
- [Directed Evolution of Proteins via Bayesian Optimization in Embedding Space](https://mlsanity.com/paper/2509.04998/) (2025)
- [Counterfactual Credit Guided Bayesian Optimization](https://mlsanity.com/paper/2510.04676/) (2025)
- [ORTHOBO: Orthogonal Bayesian Hyperparameter Optimization](https://mlsanity.com/paper/2605.06454/) (2026)

## Abstract (arXiv)

Inference in the presence of outliers is an important field of research as outliers are ubiquitous and may arise across a variety of problems and domains. Bayesian optimization is method that heavily relies on probabilistic inference. This allows outstanding sample efficiency because the probabilistic machinery provides a memory of the whole optimization process. However, that virtue becomes a disadvantage when the memory is populated with outliers, inducing bias in the estimation. In this paper, we present an empirical evaluation of Bayesian optimization methods in the presence of outliers. The empirical evidence shows that Bayesian optimization with robust regression often produces suboptimal results. We then propose a new algorithm which combines robust regression (a Gaussian process with Student-t likelihood) with outlier diagnostics to classify data points as outliers or inliers. By using an scheduler for the classification of outliers, our method is more efficient and has better convergence over the standard robust regression. Furthermore, we show that even in controlled situations with no expected outliers, our method is able to produce better results.