# Disentangling Generative Factors of Physical Fields Using Variational Autoencoders

arXiv: https://arxiv.org/abs/2109.07399 · HTML: https://mlsanity.com/paper/2109.07399/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 15 Sep 2021.

## Builds on (most cited first, 16 of 16)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [Understanding disentangling in β-VAE](https://mlsanity.com/paper/1804.03599/) (2018)
- [Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/1801.06879/) (2018)
- [Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing](https://mlsanity.com/paper/1903.10145/) (2019)
- [InfoVAE: Information Maximizing Variational Autoencoders](https://mlsanity.com/paper/1706.02262/) (2017)
- [Variational Autoencoders Pursue PCA Directions (by Accident)](https://mlsanity.com/paper/1812.06775/) (2018)
- [Learning Hierarchical Priors in VAEs](https://mlsanity.com/paper/1905.04982/) (2019)
- [Tutorial: Deriving the Standard Variational Autoencoder (VAE) Loss Function](https://mlsanity.com/paper/1907.08956/) (2019)
- [Extracting Interpretable Physical Parameters from Spatiotemporal Systems using Unsupervised Learning](https://mlsanity.com/paper/1907.06011/) (2019)
- [PDE-Driven Spatiotemporal Disentanglement](https://mlsanity.com/paper/2008.01352/) (2020)
- [A Tutorial on VAEs: From Bayes' Rule to Lossless Compression](https://mlsanity.com/paper/2006.10273/) (2020)
- [Variational Autoencoding of PDE Inverse Problems](https://mlsanity.com/paper/2006.15641/) (2020)

## Built on by (most cited first, 5 of 5)

- [CoCoGen: Physically-Consistent and Conditioned Score-based Generative Models for Forward and Inverse Problems](https://mlsanity.com/paper/2312.10527/) (2023)
- [Fully probabilistic deep models for forward and inverse problems in parametric PDEs](https://mlsanity.com/paper/2208.04856/) (2022)
- [Multiscale Graph Neural Network Autoencoders for Interpretable Scientific Machine Learning](https://mlsanity.com/paper/2302.06186/) (2023)
- [Physics-informed Variational Autoencoders for Improved Robustness to Environmental Factors of Variation](https://mlsanity.com/paper/2210.10418/) (2022)
- [Disentangled Latent Spaces for Reduced Order Models using Deterministic Autoencoders](https://mlsanity.com/paper/2502.14679/) (2025)

## Abstract (arXiv)

The ability to extract generative parameters from high-dimensional fields of data in an unsupervised manner is a highly desirable yet unrealized goal in computational physics. This work explores the use of variational autoencoders (VAEs) for non-linear dimension reduction with the specific aim of em disentangling the low-dimensional latent variables to identify independent physical parameters that generated the data. A disentangled decomposition is interpretable, and can be transferred to a variety of tasks including generative modeling, design optimization, and probabilistic reduced order modelling. A major emphasis of this work is to characterize disentanglement using VAEs while minimally modifying the classic VAE loss function (i.e. the Evidence Lower Bound) to maintain high reconstruction accuracy. The loss landscape is characterized by over-regularized local minima which surround desirable solutions. We illustrate comparisons between disentangled and entangled representations by juxtaposing learned latent distributions and the true generative factors in a model porous flow problem. Hierarchical priors are shown to facilitate the learning of disentangled representations. The regularization loss is unaffected by latent rotation when training with rotationally-invariant priors, and thus learning non-rotationally-invariant priors aids in capturing the properties of generative factors, improving disentanglement. Finally, it is shown that semi-supervised learning - accomplished by labeling a small number of samples (O(1%)) - results in accurate disentangled latent representations that can be consistently learned.