# Variational Autoencoder with Learned Latent Structure

arXiv: https://arxiv.org/abs/2006.10597 · HTML: https://mlsanity.com/paper/2006.10597/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2020.

## Built on by (most cited first, 30 of 41)

- [Pythae: Unifying Generative Autoencoders in Python -- A Benchmarking Use Case](https://mlsanity.com/paper/2206.08309/) (2022)
- [A Geometric Perspective on Variational Autoencoders](https://mlsanity.com/paper/2209.07370/) (2022)
- [Toward a Geometrical Understanding of Self-supervised Contrastive Learning](https://mlsanity.com/paper/2205.06926/) (2022)
- [Decomposed Linear Dynamical Systems (dLDS) for learning the latent components of neural dynamics](https://mlsanity.com/paper/2206.02972/) (2022)
- [Latent SDEs on Homogeneous Spaces](https://mlsanity.com/paper/2306.16248/) (2023)
- [Addressing the Topological Defects of Disentanglement via Distributed Operators](https://mlsanity.com/paper/2102.05623/) (2021)
- [Flow Factorized Representation Learning](https://mlsanity.com/paper/2309.13167/) (2023)
- [Exploring the Latent Space of Autoencoders with Interventional Assays](https://mlsanity.com/paper/2106.16091/) (2021)
- [VTAE: Variational Transformer Autoencoder with Manifolds Learning](https://mlsanity.com/paper/2304.00948/) (2023)
- [Variational Sparse Coding with Learned Thresholding](https://mlsanity.com/paper/2205.03665/) (2022)
- [MMBind: Unleashing the Potential of Distributed and Heterogeneous Data for Multimodal Learning in IoT](https://mlsanity.com/paper/2411.12126/) (2024)
- [Geometry-Preserving Encoder/Decoder in Latent Generative Models](https://mlsanity.com/paper/2501.09876/) (2025)
- [Learning Identity-Preserving Transformations on Data Manifolds](https://mlsanity.com/paper/2106.12096/) (2021)
- [Machine learning in bioprocess development: From promise to practice](https://mlsanity.com/paper/2210.02200/) (2022)
- [Robust Self-Supervised Learning with Lie Groups](https://mlsanity.com/paper/2210.13356/) (2022)
- [Generating Feasible and Plausible Counterfactual Explanations for Outcome Prediction of Business Processes](https://mlsanity.com/paper/2403.09232/) (2024)
- [Learning Latent Graph Structures and their Uncertainty](https://mlsanity.com/paper/2405.19933/) (2024)
- [Homomorphic Self-Supervised Learning](https://mlsanity.com/paper/2211.08282/) (2022)
- [Learning Internal Representations of 3D Transformations from 2D Projected Inputs](https://mlsanity.com/paper/2303.17776/) (2023)
- [Conditional and Residual Methods in Scalable Coding for Humans and Machines](https://mlsanity.com/paper/2305.02562/) (2023)
- [Unsupervised Representation Learning from Sparse Transformation Analysis](https://mlsanity.com/paper/2410.05564/) (2024)
- [Latent Space Characterization of Autoencoder Variants](https://mlsanity.com/paper/2412.04755/) (2024)
- [Riemannian Generative Decoder](https://mlsanity.com/paper/2506.19133/) (2025)
- [AR-Flow VAE: A Structured Autoregressive Flow Prior Variational Autoencoder for Unsupervised Blind Source Separation](https://mlsanity.com/paper/2603.14441/) (2026)
- [Interval Bound Interpolation for Few-shot Learning with Few Tasks](https://mlsanity.com/paper/2204.03511/) (2022)
- [Manifold Contrastive Learning with Variational Lie Group Operators](https://mlsanity.com/paper/2306.13544/) (2023)
- [Blessing of Dimensionality for Approximating Sobolev Classes on Manifolds](https://mlsanity.com/paper/2408.06996/) (2024)
- [Estimating Dataset Dimension via Singular Metrics under the Manifold Hypothesis: Application to Inverse Problems](https://mlsanity.com/paper/2507.07291/) (2025)
- [On the Adversarial Robustness of Generative Autoencoders in the Latent Space](https://mlsanity.com/paper/2307.02202/) (2023)
- [Modularity aided consistent attributed graph clustering via coarsening](https://mlsanity.com/paper/2407.07128/) (2024)

## Abstract (arXiv)

The manifold hypothesis states that high-dimensional data can be modeled as lying on or near a low-dimensional, nonlinear manifold. Variational Autoencoders (VAEs) approximate this manifold by learning mappings from low-dimensional latent vectors to high-dimensional data while encouraging a global structure in the latent space through the use of a specified prior distribution. When this prior does not match the structure of the true data manifold, it can lead to a less accurate model of the data. To resolve this mismatch, we introduce the Variational Autoencoder with Learned Latent Structure (VAELLS) which incorporates a learnable manifold model into the latent space of a VAE. This enables us to learn the nonlinear manifold structure from the data and use that structure to define a prior in the latent space. The integration of a latent manifold model not only ensures that our prior is well-matched to the data, but also allows us to define generative transformation paths in the latent space and describe class manifolds with transformations stemming from examples of each class. We validate our model on examples with known latent structure and also demonstrate its capabilities on a real-world dataset.