# Explicitly disentangling image content from translation and rotation with spatial-VAE

arXiv: https://arxiv.org/abs/1909.11663 · HTML: https://mlsanity.com/paper/1909.11663/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Sep 2019.

## Built on by (most cited first, 28 of 28)

- [Improved Transformer for High-Resolution GANs](https://mlsanity.com/paper/2106.07631/) (2021)
- [Amortized Inference for Heterogeneous Reconstruction in Cryo-EM](https://mlsanity.com/paper/2210.07387/) (2022)
- [Unsupervised Deep Learning Methods for Biological Image Reconstruction and Enhancement](https://mlsanity.com/paper/2105.08040/) (2021)
- [Physics-informed Guided Disentanglement in Generative Networks](https://mlsanity.com/paper/2107.14229/) (2021)
- [Review of Disentanglement Approaches for Medical Applications -- Towards Solving the Gordian Knot of Generative Models in Healthcare](https://mlsanity.com/paper/2203.11132/) (2022)
- [Unsupervised Object Representation Learning using Translation and Rotation Group Equivariant VAE](https://mlsanity.com/paper/2210.12918/) (2022)
- [Deep Kernel Methods Learn Better: From Cards to Process Optimization](https://mlsanity.com/paper/2303.14554/) (2023)
- [Physics and Chemistry from Parsimonious Representations: Image Analysis via Invariant Variational Autoencoders](https://mlsanity.com/paper/2303.18236/) (2023)
- [Learning Rotation Invariant Features for Cryogenic Electron Microscopy Image Reconstruction](https://mlsanity.com/paper/2101.03549/) (2021)
- [Finding simplicity: unsupervised discovery of features, patterns, and order parameters via shift-invariant variational autoencoders](https://mlsanity.com/paper/2106.12472/) (2021)
- [MELON: NeRF with Unposed Images in SO(3)](https://mlsanity.com/paper/2303.08096/) (2023)
- [Generative modeling of living cells with SO(3)-equivariant implicit neural representations](https://mlsanity.com/paper/2304.08960/) (2023)
- [Rotation and Translation Invariant Representation Learning with Implicit Neural Representations](https://mlsanity.com/paper/2304.13995/) (2023)
- [Equivariant Priors for Compressed Sensing with Unknown Orientation](https://mlsanity.com/paper/2206.14069/) (2022)
- [Integrating Predictive and Generative Capabilities by Latent Space Design via the DKL-VAE Model](https://mlsanity.com/paper/2503.02978/) (2025)
- [Cloning Deterministic Worlds: The Critical Role of Latent Geometry in Long-Horizon World Models](https://mlsanity.com/paper/2510.26782/) (2025)
- [Automated and Autonomous Experiment in Electron and Scanning Probe Microscopy](https://mlsanity.com/paper/2103.12165/) (2021)
- [Invariance-based Multi-Clustering of Latent Space Embeddings for Equivariant Learning](https://mlsanity.com/paper/2107.11717/) (2021)
- [Affinity-VAE: incorporating prior knowledge in representation learning from scientific images](https://mlsanity.com/paper/2209.04517/) (2022)
- [Equivariant amortized inference of poses for cryo-EM](https://mlsanity.com/paper/2406.01630/) (2024)
- [Reward driven discovery of the optimal microstructure representations with invariant variational autoencoders](https://mlsanity.com/paper/2510.00243/) (2025)
- [Disentangling semantic features of macromolecules in Cryo-Electron Tomography](https://mlsanity.com/paper/2106.14192/) (2021)
- [Microscopy is All You Need](https://mlsanity.com/paper/2210.06526/) (2022)
- [Domain Generalization In Robust Invariant Representation](https://mlsanity.com/paper/2304.03431/) (2023)
- [DualContrast: Unsupervised Disentangling of Content and Transformations with Implicit Parameterization](https://mlsanity.com/paper/2405.16796/) (2024)
- [Distance-Preserving Representations for Genomic Spatial Reconstruction](https://mlsanity.com/paper/2408.00911/) (2024)
- [Unsupervised SE(3) Disentanglement for in situ Macromolecular Morphology Identification from Cryo-Electron Tomography](https://mlsanity.com/paper/2601.01364/) (2026)
- [Equivariant Latent Alignment via Flow Matching under Group Symmetries](https://mlsanity.com/paper/2605.30705/) (2026)

## Abstract (arXiv)

Given an image dataset, we are often interested in finding data generative factors that encode semantic content independently from pose variables such as rotation and translation. However, current disentanglement approaches do not impose any specific structure on the learned latent representations. We propose a method for explicitly disentangling image rotation and translation from other unstructured latent factors in a variational autoencoder (VAE) framework. By formulating the generative model as a function of the spatial coordinate, we make the reconstruction error differentiable with respect to latent translation and rotation parameters. This formulation allows us to train a neural network to perform approximate inference on these latent variables while explicitly constraining them to only represent rotation and translation. We demonstrate that this framework, termed spatial-VAE, effectively learns latent representations that disentangle image rotation and translation from content and improves reconstruction over standard VAEs on several benchmark datasets, including applications to modeling continuous 2-D views of proteins from single particle electron microscopy and galaxies in astronomical images.