# SMT 2.0: A Surrogate Modeling Toolbox with a focus on Hierarchical and Mixed Variables Gaussian Processes

arXiv: https://arxiv.org/abs/2305.13998 · HTML: https://mlsanity.com/paper/2305.13998/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 97 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 22 of 22)

- [SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization](https://mlsanity.com/paper/2109.09831/) (2021)
- [Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly](https://mlsanity.com/paper/1903.06694/) (2019)
- [Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processes](https://mlsanity.com/paper/1706.03673/) (2017)
- [Active Learning of Linear Embeddings for Gaussian Processes](https://mlsanity.com/paper/1310.6740/) (2013)
- [A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factors](https://mlsanity.com/paper/1806.07504/) (2018)
- [A Systematic Exploration of Reservoir Computing for Forecasting Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/2201.08910/) (2022)
- [Group kernels for Gaussian process metamodels with categorical inputs](https://mlsanity.com/paper/1802.02368/) (2018)
- [Bayesian Optimization using Deep Gaussian Processes](https://mlsanity.com/paper/1905.03350/) (2019)
- [A mixed-categorical correlation kernel for Gaussian process](https://mlsanity.com/paper/2211.08262/) (2022)
- [Gradient-enhanced kriging for high-dimensional problems](https://mlsanity.com/paper/1708.02663/) (2017)
- [Efficient global optimization of constrained mixed variable problems](https://mlsanity.com/paper/1806.03975/) (2018)
- [Deep Gaussian Process Emulation using Stochastic Imputation](https://mlsanity.com/paper/2107.01590/) (2021)
- [Trieste: Efficiently Exploring The Depths of Black-box Functions with TensorFlow](https://mlsanity.com/paper/2302.08436/) (2023)
- [Bayesian optimization of variable-size design space problems](https://mlsanity.com/paper/2003.03300/) (2020)
- [OpenBox: A Python Toolkit for Generalized Black-box Optimization](https://mlsanity.com/paper/2304.13339/) (2023)
- [Continuous surrogate-based optimization algorithms are well-suited for expensive discrete problems](https://mlsanity.com/paper/2011.03431/) (2020)
- [A general mathematical framework for constrained mixed-variable blackbox optimization problems with meta and categorical variables](https://mlsanity.com/paper/2204.00886/) (2022)
- [A comparison of mixed-variables Bayesian optimization approaches](https://mlsanity.com/paper/2111.01533/) (2021)
- [Multioutput Gaussian Processes with Functional Data: A Study on Coastal Flood Hazard Assessment](https://mlsanity.com/paper/2007.14052/) (2020)
- [Variance based sensitivity analysis for Monte Carlo and importance sampling reliability assessment with Gaussian processes](https://mlsanity.com/paper/2011.15001/) (2020)
- [A Kernel for Hierarchical Parameter Spaces](https://mlsanity.com/paper/1310.5738/) (2013)
- [A First Analysis of Kernels for Kriging-based Optimization in Hierarchical Search Spaces](https://mlsanity.com/paper/1807.01011/) (2018)

## Built on by (most cited first, 29 of 29)

- [Aero-Nef: Neural Fields for Rapid Aircraft Aerodynamics Simulations](https://mlsanity.com/paper/2407.19916/) (2024)
- [Towards scalable surrogate models based on Neural Fields for large scale aerodynamic simulations](https://mlsanity.com/paper/2505.14704/) (2025)
- [High-dimensional mixed-categorical Gaussian processes with application to multidisciplinary design optimization for a green aircraft](https://mlsanity.com/paper/2311.06130/) (2023)
- [Geometry aware inference of steady state PDEs using Equivariant Neural Fields representations](https://mlsanity.com/paper/2504.18591/) (2025)
- [Surrogate Modeling and Explainable Artificial Intelligence for Complex Systems: A Workflow for Automated Simulation Exploration](https://mlsanity.com/paper/2510.16742/) (2025)
- [A distance for mixed-variable and hierarchical domains with meta variables](https://mlsanity.com/paper/2405.13073/) (2024)
- [SMT-EX: An Explainable Surrogate Modeling Toolbox for Mixed-Variables Design Exploration](https://mlsanity.com/paper/2503.19496/) (2025)
- [Multi-objective Bayesian Optimization With Mixed-categorical Design Variables for Expensive-to-evaluate Aeronautical Applications](https://mlsanity.com/paper/2504.09930/) (2025)
- [ONERA's CRM WBPN database for machine learning activities, related regression challenge and first results](https://mlsanity.com/paper/2505.06265/) (2025)
- [Modeling Hierarchical Spaces: A Review and Unified Framework for Surrogate-Based Architecture Design](https://mlsanity.com/paper/2506.22621/) (2025)
- [Bayesian Optimization of a Lightweight and Accurate Neural Network for Aerodynamic Performance Prediction](https://mlsanity.com/paper/2503.19479/) (2025)
- [Surrogate-based optimization of system architectures subject to hidden constraints](https://mlsanity.com/paper/2504.08721/) (2025)
- [SMRS: advocating a unified reporting standard for surrogate models in the artificial intelligence era](https://mlsanity.com/paper/2502.06753/) (2025)
- [System-of-systems Modeling and Optimization: An Integrated Framework for Intermodal Mobility](https://mlsanity.com/paper/2507.08715/) (2025)
- [A Continuous Encoding-Based Representation for Efficient Multi-Fidelity Multi-Objective Neural Architecture Search](https://mlsanity.com/paper/2509.01943/) (2025)
- [Adaptive Agents in Spatial Double-Auction Markets: Modeling the Emergence of Industrial Symbiosis](https://mlsanity.com/paper/2512.17979/) (2025)
- [Jacobian-Enhanced Neural Networks](https://mlsanity.com/paper/2406.09132/) (2024)
- [Open-Source High-Speed Flight Surrogate Modeling Framework](https://mlsanity.com/paper/2411.03598/) (2024)
- [Mod`eles de Substitution pour les Mod`eles `a base d'Agents : Enjeux, M'ethodes et Applications](https://mlsanity.com/paper/2505.11912/) (2025)
- [Frequency-aware Surrogate Modeling With SMT Kernels For Advanced Data Forecasting](https://mlsanity.com/paper/2507.09694/) (2025)
- [Global Sensitivity Analysis for Engineering Design Based on Individual Conditional Expectations](https://mlsanity.com/paper/2512.11946/) (2025)
- [Practical Efficient Global Optimization is No-regret](https://mlsanity.com/paper/2603.25311/) (2026)
- [Multi-fidelity approaches for general constrained Bayesian optimization with application to aircraft design](https://mlsanity.com/paper/2603.28987/) (2026)
- [Transfer Learning in Bayesian Optimization for Aircraft Design](https://mlsanity.com/paper/2603.28999/) (2026)
- [Robust and Explainable Divide-and-Conquer Learning for Intrusion Detection](https://mlsanity.com/paper/2605.02015/) (2026)
- [Efficient multidisciplinary design via Bayesian optimization](https://mlsanity.com/paper/2607.22560/) (2026)
- [Hierarchical Bayesian optimization of an aircraft-based multi-agent system-of-systems](https://mlsanity.com/paper/2609.22130/) (2026)
- [STREGO: A Scalable Implementation of a Trust-Region Framework for Noisy Bayesian Optimization](https://mlsanity.com/paper/2610.14338/) (2026)
- [GpFlash: Maximizing Speed and Minimizing Memory for Gaussian Process Training and Prediction](https://mlsanity.com/paper/2610.16861/) (2026)

## Abstract (arXiv)

The Surrogate Modeling Toolbox (SMT) is an open-source Python package that offers a collection of surrogate modeling methods, sampling techniques, and a set of sample problems. This paper presents SMT 2.0, a major new release of SMT that introduces significant upgrades and new features to the toolbox. This release adds the capability to handle mixed-variable surrogate models and hierarchical variables. These types of variables are becoming increasingly important in several surrogate modeling applications. SMT 2.0 also improves SMT by extending sampling methods, adding new surrogate models, and computing variance and kernel derivatives for Kriging. This release also includes new functions to handle noisy and use multifidelity data. To the best of our knowledge, SMT 2.0 is the first open-source surrogate library to propose surrogate models for hierarchical and mixed inputs. This open-source software is distributed under the New BSD license.