# Scalable Gaussian Processes for Data-Driven Design using Big Data with Categorical Factors

arXiv: https://arxiv.org/abs/2106.15356 · HTML: https://mlsanity.com/paper/2106.15356/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 51 references that are in the map.
- First posted on arXiv 26 Jun 2021.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)](https://mlsanity.com/paper/1503.01057/) (2015)
- [GPflow: A Gaussian process library using TensorFlow](https://mlsanity.com/paper/1610.08733/) (2016)
- [A Comparative Analysis of XGBoost](https://mlsanity.com/paper/1911.01914/) (2019)
- [Local Gaussian process approximation for large computer experiments](https://mlsanity.com/paper/1303.0383/) (2013)
- [Deep Generative Modeling for Mechanistic-based Learning and Design of Metamaterial Systems](https://mlsanity.com/paper/2006.15274/) (2020)
- [A Framework for Interdomain and Multioutput Gaussian Processes](https://mlsanity.com/paper/2003.01115/) (2020)
- [Natural Gradients in Practice: Non-Conjugate Variational Inference in Gaussian Process Models](https://mlsanity.com/paper/1803.09151/) (2018)
- [A deep material network for multiscale topology learning and accelerated nonlinear modeling of heterogeneous materials](https://mlsanity.com/paper/1807.09829/) (2018)
- [Reliable and Explainable Machine Learning Methods for Accelerated Material Discovery](https://mlsanity.com/paper/1901.02717/) (2019)
- [Data-Driven Topology Optimization with Multiclass Microstructures using Latent Variable Gaussian Process](https://mlsanity.com/paper/2006.15273/) (2020)

## Built on by (most cited first, 11 of 11)

- [Multi-Fidelity Cost-Aware Bayesian Optimization](https://mlsanity.com/paper/2211.02732/) (2022)
- [Data-Driven Design for Metamaterials and Multiscale Systems: A Review](https://mlsanity.com/paper/2307.05506/) (2023)
- [A Latent Variable Approach for Non-Hierarchical Multi-Fidelity Adaptive Sampling](https://mlsanity.com/paper/2310.03298/) (2023)
- [A Gaussian Process Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations](https://mlsanity.com/paper/2401.03492/) (2024)
- [Uncertainty-Aware Mixed-Variable Machine Learning for Materials Design](https://mlsanity.com/paper/2207.04994/) (2022)
- [Improving Hyperparameter Optimization with Checkpointed Model Weights](https://mlsanity.com/paper/2406.18630/) (2024)
- [Interpretable Multi-Source Data Fusion Through Latent Variable Gaussian Process](https://mlsanity.com/paper/2402.04146/) (2024)
- [Image-based Artificial Intelligence empowered surrogate model and shape morpher for real-time blank shape optimisation in the hot stamping process](https://mlsanity.com/paper/2212.05885/) (2022)
- [Scalable mixed-domain Gaussian process modeling and model reduction for longitudinal data](https://mlsanity.com/paper/2111.02019/) (2021)
- [Forward variable selection enables fast and accurate dynamic system identification with Karhunen-Lo`eve decomposed Gaussian processes](https://mlsanity.com/paper/2205.13676/) (2022)
- [Emerging Microelectronic Materials by Design: Navigating Combinatorial Design Space with Scarce and Dispersed Data](https://mlsanity.com/paper/2412.17283/) (2024)

## Abstract (arXiv)

Scientific and engineering problems often require the use of artificial intelligence to aid understanding and the search for promising designs. While Gaussian processes (GP) stand out as easy-to-use and interpretable learners, they have difficulties in accommodating big datasets, categorical inputs, and multiple responses, which has become a common challenge for a growing number of data-driven design applications. In this paper, we propose a GP model that utilizes latent variables and functions obtained through variational inference to address the aforementioned challenges simultaneously. The method is built upon the latent variable Gaussian process (LVGP) model where categorical factors are mapped into a continuous latent space to enable GP modeling of mixed-variable datasets. By extending variational inference to LVGP models, the large training dataset is replaced by a small set of inducing points to address the scalability issue. Output response vectors are represented by a linear combination of independent latent functions, forming a flexible kernel structure to handle multiple responses that might have distinct behaviors. Comparative studies demonstrate that the proposed method scales well for large datasets with over 10^4 data points, while outperforming state-of-the-art machine learning methods without requiring much hyperparameter tuning. In addition, an interpretable latent space is obtained to draw insights into the effect of categorical factors, such as those associated with building blocks of architectures and element choices in metamaterial and materials design. Our approach is demonstrated for machine learning of ternary oxide materials and topology optimization of a multiscale compliant mechanism with aperiodic microstructures and multiple materials.