# A distance for mixed-variable and hierarchical domains with meta variables

arXiv: https://arxiv.org/abs/2405.13073 · HTML: https://mlsanity.com/paper/2405.13073/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 52 references that are in the map.
- First posted on arXiv 20 May 2024.

## Builds on (most cited first, 14 of 14)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice](https://mlsanity.com/paper/2007.15745/) (2020)
- [Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processes](https://mlsanity.com/paper/1706.03673/) (2017)
- [A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factors](https://mlsanity.com/paper/1806.07504/) (2018)
- [SMT 2.0: A Surrogate Modeling Toolbox with a focus on Hierarchical and Mixed Variables Gaussian Processes](https://mlsanity.com/paper/2305.13998/) (2023)
- [A mixed-categorical correlation kernel for Gaussian process](https://mlsanity.com/paper/2211.08262/) (2022)
- [Efficient global optimization of constrained mixed variable problems](https://mlsanity.com/paper/1806.03975/) (2018)
- [Bayesian optimization of variable-size design space problems](https://mlsanity.com/paper/2003.03300/) (2020)
- [A general mathematical framework for constrained mixed-variable blackbox optimization problems with meta and categorical variables](https://mlsanity.com/paper/2204.00886/) (2022)
- [A Kernel for Hierarchical Parameter Spaces](https://mlsanity.com/paper/1310.5738/) (2013)
- [On the Definiteness of Earth Mover's Distance and Its Relation to Set Intersection](https://mlsanity.com/paper/1510.02833/) (2015)
- [HyperNOMAD: Hyperparameter optimization of deep neural networks using mesh adaptive direct search](https://mlsanity.com/paper/1907.01698/) (2019)
- [High-dimensional multidisciplinary design optimization for aircraft eco-design / Optimisation multi-disciplinaire en grande dimension pour l''eco-conception avion en avant-projet](https://mlsanity.com/paper/2402.04711/) (2024)
- [Graph Neural Distance Metric Learning with Graph-Bert](https://mlsanity.com/paper/2002.03427/) (2020)

## Built on by (most cited first, 5 of 5)

- [Modeling Hierarchical Spaces: A Review and Unified Framework for Surrogate-Based Architecture Design](https://mlsanity.com/paper/2506.22621/) (2025)
- [Bayesian Optimization of a Lightweight and Accurate Neural Network for Aerodynamic Performance Prediction](https://mlsanity.com/paper/2503.19479/) (2025)
- [Efficient multidisciplinary design via Bayesian optimization](https://mlsanity.com/paper/2607.22560/) (2026)
- [Hierarchical Bayesian optimization of an aircraft-based multi-agent system-of-systems](https://mlsanity.com/paper/2609.22130/) (2026)
- [Energy-aware frugal Bayesian optimization](https://mlsanity.com/paper/2609.31638/) (2026)

## Abstract (arXiv)

Heterogeneous datasets emerge in various machine learning and optimization applications that feature different input sources, types or formats. Most models or methods do not natively tackle heterogeneity. Hence, such datasets are often partitioned into smaller and simpler ones, which may limit the generalizability or performance, especially when data is limited. The first main contribution of this work is a modeling framework that generalizes hierarchical, tree-structured, variable-size or conditional search frameworks. The framework models mixed-variable and hierarchical domains in which variables may be continuous, integer, or categorical, with some identified as meta when they influence the structure of the problem. The second main contribution is a novel distance that compares any pair of mixed-variable points that do not share the same variables, allowing to use whole heterogeneous datasets that reside in mixed-variable and hierarchical domains with meta variables. The contributions are illustrated through regression and classification experiments using simple distance-based models applied to datasets of hyperparameters with corresponding performance scores.