# Input Warping for Bayesian Optimization of Non-stationary Functions

arXiv: https://arxiv.org/abs/1402.0929 · HTML: https://mlsanity.com/paper/1402.0929/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Feb 2014.

## Built on by (most cited first, 30 of 67)

- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges](https://mlsanity.com/paper/2107.05847/) (2021)
- [Vanilla Bayesian Optimization Performs Great in High Dimensions](https://mlsanity.com/paper/2402.02229/) (2024)
- [Multi-Objective Bayesian Optimization over High-Dimensional Search Spaces](https://mlsanity.com/paper/2109.10964/) (2021)
- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [PFNs4BO: In-Context Learning for Bayesian Optimization](https://mlsanity.com/paper/2305.17535/) (2023)
- [πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization](https://mlsanity.com/paper/2204.11051/) (2022)
- [High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning](https://mlsanity.com/paper/2106.03609/) (2021)
- [Looping in the Human Collaborative and Explainable Bayesian Optimization](https://mlsanity.com/paper/2310.17273/) (2023)
- [Completed Hyperparameter Transfer across Modules, Width, Depth, Batch and Duration](https://mlsanity.com/paper/2512.22382/) (2025)
- [BORE: Bayesian Optimization by Density-Ratio Estimation](https://mlsanity.com/paper/2102.09009/) (2021)
- [Diffusion Model for Data-Driven Black-Box Optimization](https://mlsanity.com/paper/2403.13219/) (2024)
- [Toward Automated Algorithm Design: A Survey and Practical Guide to Meta-Black-Box-Optimization](https://mlsanity.com/paper/2411.00625/) (2024)
- [Self-Correcting Bayesian Optimization through Bayesian Active Learning](https://mlsanity.com/paper/2304.11005/) (2023)
- [Bag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimization](https://mlsanity.com/paper/2105.01015/) (2021)
- [Statistical Deep Learning for Spatial and Spatio-Temporal Data](https://mlsanity.com/paper/2206.02218/) (2022)
- [Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processes](https://mlsanity.com/paper/2203.01459/) (2022)
- [qPOTS: Efficient batch multiobjective Bayesian optimization via Pareto optimal Thompson sampling](https://mlsanity.com/paper/2310.15788/) (2023)
- [Efficient Black-box Adversarial Attacks via Bayesian Optimization Guided by a Function Prior](https://mlsanity.com/paper/2405.19098/) (2024)
- [Revisiting Bayesian Optimization in the light of the COCO benchmark](https://mlsanity.com/paper/2103.16649/) (2021)
- [Online Calibrated and Conformal Prediction Improves Bayesian Optimization](https://mlsanity.com/paper/2112.04620/) (2021)
- [SCOOT: SLO-Oriented Performance Tuning for LLM Inference Engines](https://mlsanity.com/paper/2408.04323/) (2024)
- [Non-smooth Bayesian Optimization in Tuning Problems](https://mlsanity.com/paper/2109.07563/) (2021)
- [Adaptive Gradient Methods with Local Guarantees](https://mlsanity.com/paper/2203.01400/) (2022)
- [Efficient Transformed Gaussian Processes for Non-Stationary Dependent Multi-class Classification](https://mlsanity.com/paper/2205.15008/) (2022)
- [How Bayesian Should Bayesian Optimisation Be?](https://mlsanity.com/paper/2105.00894/) (2021)
- [Unleashing the Potential of Acquisition Functions in High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2302.08298/) (2023)
- [Tailoring Mixup to Data for Calibration](https://mlsanity.com/paper/2311.01434/) (2023)
- [Heuristic-free Optimization of Force-Controlled Robot Search Strategies in Stochastic Environments](https://mlsanity.com/paper/2207.07524/) (2022)
- [Hierarchical-Hyperplane Kernels for Actively Learning Gaussian Process Models of Nonstationary Systems](https://mlsanity.com/paper/2303.10022/) (2023)

## Abstract (arXiv)

Bayesian optimization has proven to be a highly effective methodology for the global optimization of unknown, expensive and multimodal functions. The ability to accurately model distributions over functions is critical to the effectiveness of Bayesian optimization. Although Gaussian processes provide a flexible prior over functions which can be queried efficiently, there are various classes of functions that remain difficult to model. One of the most frequently occurring of these is the class of non-stationary functions. The optimization of the hyperparameters of machine learning algorithms is a problem domain in which parameters are often manually transformed a priori, for example by optimizing in "log-space," to mitigate the effects of spatially-varying length scale. We develop a methodology for automatically learning a wide family of bijective transformations or warpings of the input space using the Beta cumulative distribution function. We further extend the warping framework to multi-task Bayesian optimization so that multiple tasks can be warped into a jointly stationary space. On a set of challenging benchmark optimization tasks, we observe that the inclusion of warping greatly improves on the state-of-the-art, producing better results faster and more reliably.