# Geometry-Preserving Encoder/Decoder in Latent Generative Models

arXiv: https://arxiv.org/abs/2501.09876 · HTML: https://mlsanity.com/paper/2501.09876/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 29 of its 60 references that are in the map.
- First posted on arXiv 16 Jan 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 29 of 29)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)
- [Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Condition](https://mlsanity.com/paper/1608.04636/) (2016)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [Regularisation of Neural Networks by Enforcing Lipschitz Continuity](https://mlsanity.com/paper/1804.04368/) (2018)
- [Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks](https://mlsanity.com/paper/1906.04893/) (2019)
- [Optimal transport mapping via input convex neural networks](https://mlsanity.com/paper/1908.10962/) (2019)
- [VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning](https://mlsanity.com/paper/1705.07761/) (2017)
- [Flow Matching in Latent Space](https://mlsanity.com/paper/2307.08698/) (2023)
- [Understanding and Mitigating Exploding Inverses in Invertible Neural Networks](https://mlsanity.com/paper/2006.09347/) (2020)
- [Sliced Gromov-Wasserstein](https://mlsanity.com/paper/1905.10124/) (2019)
- [Learning Generative Models across Incomparable Spaces](https://mlsanity.com/paper/1905.05461/) (2019)
- [Variational Autoencoder with Learned Latent Structure](https://mlsanity.com/paper/2006.10597/) (2020)
- [Embrace the Gap: VAEs Perform Independent Mechanism Analysis](https://mlsanity.com/paper/2206.02416/) (2022)
- [PaGoDA: Progressive Growing of a One-Step Generator from a Low-Resolution Diffusion Teacher](https://mlsanity.com/paper/2405.14822/) (2024)
- [Isometric Autoencoders](https://mlsanity.com/paper/2006.09289/) (2020)
- [DyDiff-VAE: A Dynamic Variational Framework for Information Diffusion Prediction](https://mlsanity.com/paper/2106.03251/) (2021)
- [On Probabilistic Embeddings in Optimal Dimension Reduction](https://mlsanity.com/paper/2408.02433/) (2024)
- [Convergent autoencoder approximation of low bending and low distortion manifold embeddings](https://mlsanity.com/paper/2208.10193/) (2022)
- [Monotone Generative Modeling via a Gromov-Monge Embedding](https://mlsanity.com/paper/2311.01375/) (2023)
- [Improving Hyperbolic Representations via Gromov-Wasserstein Regularization](https://mlsanity.com/paper/2407.10495/) (2024)

## Built on by (most cited first, 6 of 6)

- [Riemannian Archetypal Analysis: Interpretable non-linear data analysis on deformed star distributions](https://mlsanity.com/paper/2605.24113/) (2026)
- [Sparsification and Reconstruction from the Perspective of Representation Geometry](https://mlsanity.com/paper/2505.22506/) (2025)
- [Privacy-Utility Trade-off in Data Publication: A Bilevel Optimization Framework with Curvature-Guided Perturbation](https://mlsanity.com/paper/2509.02048/) (2025)
- [Complex variational autoencoders admit K"ahler structure](https://mlsanity.com/paper/2511.15172/) (2025)
- [Understanding Latent Diffusability via Fisher Geometry](https://mlsanity.com/paper/2604.02751/) (2026)
- [Continuous-Time Trajectory Generation from Discrete Observations with Stochasticity](https://mlsanity.com/paper/2609.32026/) (2026)

## Abstract (arXiv)

Generative modeling aims to generate new data samples that resemble a given dataset. When using diffusion models for this task, one of the main challenges is solving the problem in the input space, which tends to be very high-dimensional. To address this, recent approaches solve diffusion models in the latent space through an encoder that maps from the data space to a lower-dimensional latent space, improving training efficiency and achieving state-of-the-art results. The variational autoencoder (VAE) is the most commonly used encoder/decoder framework in this domain, known for its ability to learn latent representations and generate data samples. In this paper, we introduce a novel encoder/decoder framework with theoretical properties distinct from those of the VAE, specifically designed to preserve the geometric structure of the data distribution. We demonstrate the significant advantages of this geometry-preserving encoder in the training process of both the encoder and decoder. Additionally, we provide theoretical results proving convergence of the training process, including convergence guarantees for encoder training, and results showing faster convergence of decoder training when using the geometry-preserving encoder.