# Reward driven discovery of the optimal microstructure representations with invariant variational autoencoders

arXiv: https://arxiv.org/abs/2510.00243 · HTML: https://mlsanity.com/paper/2510.00243/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Builds on (most cited first, 16 of 16)

- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Neural Architecture Optimization](https://mlsanity.com/paper/1808.07233/) (2018)
- [Multi-Objective Hyperparameter Optimization in Machine Learning -- An Overview](https://mlsanity.com/paper/2206.07438/) (2022)
- [Explicitly disentangling image content from translation and rotation with spatial-VAE](https://mlsanity.com/paper/1909.11663/) (2019)
- [A survey on multi-objective hyperparameter optimization algorithms for Machine Learning](https://mlsanity.com/paper/2111.13755/) (2021)
- [Overview: Computer vision and machine learning for microstructural characterization and analysis](https://mlsanity.com/paper/2005.14260/) (2020)
- [Deep Kernel Methods Learn Better: From Cards to Process Optimization](https://mlsanity.com/paper/2303.14554/) (2023)
- [Physics and Chemistry from Parsimonious Representations: Image Analysis via Invariant Variational Autoencoders](https://mlsanity.com/paper/2303.18236/) (2023)
- [Robust Feature Disentanglement in Imaging Data via Joint Invariant Variational Autoencoders: from Cards to Atoms](https://mlsanity.com/paper/2104.10180/) (2021)
- [Unraveling the Impact of Initial Choices and In-Loop Interventions on Learning Dynamics in Autonomous Scanning Probe Microscopy](https://mlsanity.com/paper/2402.00071/) (2024)
- [Learning Metal Microstructural Heterogeneity through Spatial Mapping of Diffraction Latent Space Features](https://mlsanity.com/paper/2501.18064/) (2025)
- [Physics-based reward driven image analysis in microscopy](https://mlsanity.com/paper/2404.14146/) (2024)
- [Unsupervised Reward-Driven Image Segmentation in Automated Scanning Transmission Electron Microscopy Experiments](https://mlsanity.com/paper/2409.12462/) (2024)
- [Rewards-based image analysis in microscopy](https://mlsanity.com/paper/2502.18522/) (2025)
- [The Power of the Pareto Front: Balancing Uncertain Rewards for Adaptive Experimentation in scanning probe microscopy](https://mlsanity.com/paper/2504.06525/) (2025)

## Built on by (most cited first, 1 of 1)

- [PATHFINDER: Multi-objective discovery in structural and spectral spaces](https://mlsanity.com/paper/2604.04194/) (2026)

## Abstract (arXiv)

Microscopy techniques generate vast amounts of complex image data that in principle can be used to discover simpler, interpretable, and parsimonious forms to reveal the underlying physical structures, such as elementary building blocks in molecular systems or order parameters and phases in crystalline materials. Variational Autoencoders (VAEs) provide a powerful means of constructing such low-dimensional representations, but their performance heavily depends on multiple non-myopic design choices, which are often optimized through trial-and-error and empirical analysis. To enable automated and unbiased optimization of VAE workflows, we investigated reward-based strategies for evaluating latent space representations. Using Piezoresponse Force Microscopy data as a model system, we examined multiple policies and reward functions that can serve as a foundation for automated optimization. Our analysis shows that approximating the latent space with Gaussian Mixture Models (GMM) and Bayesian Gaussian Mixture Models (BGMM) provides a strong basis for constructing reward functions capable of estimating model efficiency and guiding the search for optimal parsimonious representations.