# Combining Variational Autoencoders and Physical Bias for Improved Microscopy Data Analysis

arXiv: https://arxiv.org/abs/2302.04216 · HTML: https://mlsanity.com/paper/2302.04216/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 56 references that are in the map.
- First posted on arXiv 8 Feb 2023.

## Builds on (most cited first, 9 of 9)

- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Balancing reconstruction error and Kullback-Leibler divergence in Variational Autoencoders](https://mlsanity.com/paper/2002.07514/) (2020)
- [AtomAI: A Deep Learning Framework for Analysis of Image and Spectroscopy Data in (Scanning) Transmission Electron Microscopy and Beyond](https://mlsanity.com/paper/2105.07485/) (2021)
- [On the Importance of the Kullback-Leibler Divergence Term in Variational Autoencoders for Text Generation](https://mlsanity.com/paper/1909.13668/) (2019)
- [Optimizing Training Trajectories in Variational Autoencoders via Latent Bayesian Optimization Approach](https://mlsanity.com/paper/2207.00128/) (2022)
- [Robust Feature Disentanglement in Imaging Data via Joint Invariant Variational Autoencoders: from Cards to Atoms](https://mlsanity.com/paper/2104.10180/) (2021)
- [Finding simplicity: unsupervised discovery of features, patterns, and order parameters via shift-invariant variational autoencoders](https://mlsanity.com/paper/2106.12472/) (2021)
- [Semi-supervised learning of images with strong rotational disorder: assembling nanoparticle libraries](https://mlsanity.com/paper/2105.11475/) (2021)
- [Semi-supervised machine learning model for analysis of nanowire morphologies from transmission electron microscopy images](https://mlsanity.com/paper/2203.13875/) (2022)

## Built on by (most cited first, 5 of 5)

- [DONUT: Physics-aware Machine Learning for Real-time X-ray Nanodiffraction Analysis](https://mlsanity.com/paper/2507.14038/) (2025)
- [Physically-Constrained Autoencoder-Assisted Bayesian Optimization for Refinement of High-Dimensional Defect-Sensitive Single Crystalline Structure](https://mlsanity.com/paper/2601.00855/) (2025)
- [Detecting Atomic Scale Surface Defects in STM of TMDs with Ensemble Deep Learning](https://mlsanity.com/paper/2312.05160/) (2023)
- [Total Variation-Based Image Decomposition and Denoising for Microscopy Images](https://mlsanity.com/paper/2505.08843/) (2025)
- [AI-Driven Phase Identification from X-ray Hyperspectral Imaging of cycled Na-ion Cathode Materials](https://mlsanity.com/paper/2603.07666/) (2026)

## Abstract (arXiv)

Electron and scanning probe microscopy produce vast amounts of data in the form of images or hyperspectral data, such as EELS or 4D STEM, that contain information on a wide range of structural, physical, and chemical properties of materials. To extract valuable insights from these data, it is crucial to identify physically separate regions in the data, such as phases, ferroic variants, and boundaries between them. In order to derive an easily interpretable feature analysis, combining with well-defined boundaries in a principled and unsupervised manner, here we present a physics augmented machine learning method which combines the capability of Variational Autoencoders to disentangle factors of variability within the data and the physics driven loss function that seeks to minimize the total length of the discontinuities in images corresponding to latent representations. Our method is applied to various materials, including NiO-LSMO, BiFeO3, and graphene. The results demonstrate the effectiveness of our approach in extracting meaningful information from large volumes of imaging data. The fully notebook containing implementation of the code and analysis workflow is available at https://github.com/arpanbiswas52/PaperNotebooks