# Physics and Chemistry from Parsimonious Representations: Image Analysis via Invariant Variational Autoencoders

arXiv: https://arxiv.org/abs/2303.18236 · HTML: https://mlsanity.com/paper/2303.18236/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 61 references that are in the map.
- First posted on arXiv 30 Mar 2023.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [Natural Graph Networks](https://mlsanity.com/paper/2007.08349/) (2020)
- [Explicitly disentangling image content from translation and rotation with spatial-VAE](https://mlsanity.com/paper/1909.11663/) (2019)
- [AtomAI: A Deep Learning Framework for Analysis of Image and Spectroscopy Data in (Scanning) Transmission Electron Microscopy and Beyond](https://mlsanity.com/paper/2105.07485/) (2021)
- [On the Usefulness of Deep Ensemble Diversity for Out-of-Distribution Detection](https://mlsanity.com/paper/2207.07517/) (2022)
- [Optimizing Training Trajectories in Variational Autoencoders via Latent Bayesian Optimization Approach](https://mlsanity.com/paper/2207.00128/) (2022)
- [Decoding the shift-invariant data: applications for band-excitation scanning probe microscopy](https://mlsanity.com/paper/2104.10207/) (2021)

## Built on by (most cited first, 7 of 7)

- [Learning Metal Microstructural Heterogeneity through Spatial Mapping of Diffraction Latent Space Features](https://mlsanity.com/paper/2501.18064/) (2025)
- [Reward driven workflows for unsupervised explainable analysis of phases and ferroic variants from atomically resolved imaging data](https://mlsanity.com/paper/2411.12612/) (2024)
- [Disentanglement by means of action-induced representations](https://mlsanity.com/paper/2602.06741/) (2026)
- [Reward driven discovery of the optimal microstructure representations with invariant variational autoencoders](https://mlsanity.com/paper/2510.00243/) (2025)
- [Unlocking Latent Dimensions: Exploring Representations of Large-Scale X-ray Scattering Data using Variational Autoencoders](https://mlsanity.com/paper/2606.14999/) (2026)
- [Proof-of-Authorship for Diffusion-based AI Generated Content](https://mlsanity.com/paper/2603.17513/) (2026)
- [AI-interpreted Optical Scattering for Robust and Focal Depth-Aware Imaging](https://mlsanity.com/paper/2607.22867/) (2026)

## Abstract (arXiv)

Electron, optical, and scanning probe microscopy methods are generating ever increasing volume of image data containing information on atomic and mesoscale structures and functionalities. This necessitates the development of the machine learning methods for discovery of physical and chemical phenomena from the data, such as manifestations of symmetry breaking in electron and scanning tunneling microscopy images, variability of the nanoparticles. Variational autoencoders (VAEs) are emerging as a powerful paradigm for the unsupervised data analysis, allowing to disentangle the factors of variability and discover optimal parsimonious representation. Here, we summarize recent developments in VAEs, covering the basic principles and intuition behind the VAEs. The invariant VAEs are introduced as an approach to accommodate scale and translation invariances present in imaging data and separate known factors of variations from the ones to be discovered. We further describe the opportunities enabled by the control over VAE architecture, including conditional, semi-supervised, and joint VAEs. Several case studies of VAE applications for toy models and experimental data sets in Scanning Transmission Electron Microscopy are discussed, emphasizing the deep connection between VAE and basic physical principles. All the codes used here are available at https://github.com/saimani5/VAE-tutorials and this article can be used as an application guide when applying these to own data sets.