# Deep Kernel Methods Learn Better: From Cards to Process Optimization

arXiv: https://arxiv.org/abs/2303.14554 · HTML: https://mlsanity.com/paper/2303.14554/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 70 references that are in the map.
- First posted on arXiv 25 Mar 2023.

## Builds on (most cited first, 19 of 19)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Grammar Variational Autoencoder](https://mlsanity.com/paper/1703.01925/) (2017)
- [Understanding Convolution for Semantic Segmentation](https://mlsanity.com/paper/1702.08502/) (2017)
- [Variational Autoencoder for Deep Learning of Images, Labels and Captions](https://mlsanity.com/paper/1609.08976/) (2016)
- [An Uncertain Future: Forecasting from Static Images using Variational Autoencoders](https://mlsanity.com/paper/1606.07873/) (2016)
- [Deep Active Learning for Efficient Training of a LiDAR 3D Object Detector](https://mlsanity.com/paper/1901.10609/) (2019)
- [Explicitly disentangling image content from translation and rotation with spatial-VAE](https://mlsanity.com/paper/1909.11663/) (2019)
- [Good practices for Bayesian Optimization of high dimensional structured spaces](https://mlsanity.com/paper/2012.15471/) (2020)
- [Weather and Light Level Classification for Autonomous Driving: Dataset, Baseline and Active Learning](https://mlsanity.com/paper/2104.14042/) (2021)
- [Towards a Theoretical Understanding of the Robustness of Variational Autoencoders](https://mlsanity.com/paper/2007.07365/) (2020)
- [Unsupervised Real Image Super-Resolution via Generative Variational AutoEncoder](https://mlsanity.com/paper/2004.12811/) (2020)
- [Exploring Diversity-based Active Learning for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/2205.07708/) (2022)
- [Bayesian Active Learning for Scanning Probe Microscopy: from Gaussian Processes to Hypothesis Learning](https://mlsanity.com/paper/2205.15458/) (2022)
- [Regression markets and application to energy forecasting](https://mlsanity.com/paper/2110.03633/) (2021)
- [A Thermodynamics-informed Active Learning Approach to Perception and Reasoning about Fluids](https://mlsanity.com/paper/2203.05775/) (2022)
- [Active learning in open experimental environments: selecting the right information channel(s) based on predictability in deep kernel learning](https://mlsanity.com/paper/2203.10181/) (2022)
- [Discovery of structure-property relations for molecules via hypothesis-driven active learning over the chemical space](https://mlsanity.com/paper/2301.02665/) (2023)

## Built on by (most cited first, 7 of 7)

- [Bayesian Co-navigation: Dynamic Designing of the Materials Digital Twins via Active Learning](https://mlsanity.com/paper/2404.12899/) (2024)
- [Measurements with Noise: Bayesian Optimization for Co-optimizing Noise and Property Discovery in Automated Experiments](https://mlsanity.com/paper/2410.02717/) (2024)
- [Integrating Predictive and Generative Capabilities by Latent Space Design via the DKL-VAE Model](https://mlsanity.com/paper/2503.02978/) (2025)
- [Reward driven discovery of the optimal microstructure representations with invariant variational autoencoders](https://mlsanity.com/paper/2510.00243/) (2025)
- [Closed-loop discovery of out-of-distribution processing protocols by evolutionary search and uncertainty-aware learning](https://mlsanity.com/paper/2606.13859/) (2026)
- [DIVIDE: A Framework for Learning from Independent Multi-Mechanism Data Using Deep Encoders and Gaussian Processes](https://mlsanity.com/paper/2511.12745/) (2025)
- [Novelty-Driven Target-Space Discovery in Automated Electron and Scanning Probe Microscopy](https://mlsanity.com/paper/2603.16715/) (2026)

## Abstract (arXiv)

The ability of deep learning methods to perform classification and regression tasks relies heavily on their capacity to uncover manifolds in high-dimensional data spaces and project them into low-dimensional representation spaces. In this study, we investigate the structure and character of the manifolds generated by classical variational autoencoder (VAE) approaches and deep kernel learning (DKL). In the former case, the structure of the latent space is determined by the properties of the input data alone, while in the latter, the latent manifold forms as a result of an active learning process that balances the data distribution and target functionalities. We show that DKL with active learning can produce a more compact and smooth latent space which is more conducive to optimization compared to previously reported methods, such as the VAE. We demonstrate this behavior using a simple cards data set and extend it to the optimization of domain-generated trajectories in physical systems. Our findings suggest that latent manifolds constructed through active learning have a more beneficial structure for optimization problems, especially in feature-rich target-poor scenarios that are common in domain sciences, such as materials synthesis, energy storage, and molecular discovery. The jupyter notebooks that encapsulate the complete analysis accompany the article.