# Finding simplicity: unsupervised discovery of features, patterns, and order parameters via shift-invariant variational autoencoders

arXiv: https://arxiv.org/abs/2106.12472 · HTML: https://mlsanity.com/paper/2106.12472/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 56 references that are in the map.
- First posted on arXiv 23 Jun 2021.

## Builds on (most cited first, 11 of 11)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019)
- [Towards a Definition of Disentangled Representations](https://mlsanity.com/paper/1812.02230/) (2018)
- [Exploring the Landscape of Spatial Robustness](https://mlsanity.com/paper/1712.02779/) (2017)
- [Disentangling Factors of Variation Using Few Labels](https://mlsanity.com/paper/1905.01258/) (2019)
- [Explicitly disentangling image content from translation and rotation with spatial-VAE](https://mlsanity.com/paper/1909.11663/) (2019)
- [Unsupervised Unmixing of Hyperspectral Images Accounting for Endmember Variability](https://mlsanity.com/paper/1406.5071/) (2014)
- [Robust Feature Disentanglement in Imaging Data via Joint Invariant Variational Autoencoders: from Cards to Atoms](https://mlsanity.com/paper/2104.10180/) (2021)
- [Ensemble learning and iterative training (ELIT) machine learning: applications towards uncertainty quantification and automated experiment in atom-resolved microscopy](https://mlsanity.com/paper/2101.08449/) (2021)

## Built on by (most cited first, 4 of 4)

- [Optimizing Training Trajectories in Variational Autoencoders via Latent Bayesian Optimization Approach](https://mlsanity.com/paper/2207.00128/) (2022)
- [From latent dynamics to meaningful representations](https://mlsanity.com/paper/2209.00905/) (2022)
- [Combining Variational Autoencoders and Physical Bias for Improved Microscopy Data Analysis](https://mlsanity.com/paper/2302.04216/) (2023)
- [Integrating Predictive and Generative Capabilities by Latent Space Design via the DKL-VAE Model](https://mlsanity.com/paper/2503.02978/) (2025)

## Abstract (arXiv)

Recent advances in scanning tunneling and transmission electron microscopies (STM and STEM) have allowed routine generation of large volumes of imaging data containing information on the structure and functionality of materials. The experimental data sets contain signatures of long-range phenomena such as physical order parameter fields, polarization and strain gradients in STEM, or standing electronic waves and carrier-mediated exchange interactions in STM, all superimposed onto scanning system distortions and gradual changes of contrast due to drift and/or mis-tilt effects. Correspondingly, while the human eye can readily identify certain patterns in the images such as lattice periodicities, repeating structural elements, or microstructures, their automatic extraction and classification are highly non-trivial and universal pathways to accomplish such analyses are absent. We pose that the most distinctive elements of the patterns observed in STM and (S)TEM images are similarity and (almost-) periodicity, behaviors stemming directly from the parsimony of elementary atomic structures, superimposed on the gradual changes reflective of order parameter distributions. However, the discovery of these elements via global Fourier methods is non-trivial due to variability and lack of ideal discrete translation symmetry. To address this problem, we develop shift-invariant variational autoencoders (shift-VAE) that allow disentangling characteristic repeating features in the images, their variations, and shifts inevitable for random sampling of image space. Shift-VAEs balance the uncertainty in the position of the object of interest with the uncertainty in shape reconstruction. This approach is illustrated for model 1D data, and further extended to synthetic and experimental STM and STEM 2D data.