# The Geometry of Deep Generative Image Models and its Applications

arXiv: https://arxiv.org/abs/2101.06006 · HTML: https://mlsanity.com/paper/2101.06006/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 7 references that are in the map.
- First posted on arXiv 15 Jan 2021.

## Builds on (most cited first, 4 of 4)

- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Large Scale Adversarial Representation Learning](https://mlsanity.com/paper/1907.02544/) (2019)
- [Latent Space Oddity: on the Curvature of Deep Generative Models](https://mlsanity.com/paper/1710.11379/) (2017)
- [Metrics for Deep Generative Models](https://mlsanity.com/paper/1711.01204/) (2017)

## Built on by (most cited first, 30 of 35)

- [Low-Rank Subspaces in GANs](https://mlsanity.com/paper/2106.04488/) (2021)
- [Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation](https://mlsanity.com/paper/2108.07668/) (2021)
- [Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANs](https://mlsanity.com/paper/2106.06959/) (2021)
- [One Model to Edit Them All: Free-Form Text-Driven Image Manipulation with Semantic Modulations](https://mlsanity.com/paper/2210.07883/) (2022)
- [Region-Based Semantic Factorization in GANs](https://mlsanity.com/paper/2202.09649/) (2022)
- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)
- [Feature-Style Encoder for Style-Based GAN Inversion](https://mlsanity.com/paper/2202.02183/) (2022)
- [Feature Unlearning for Pre-trained GANs and VAEs](https://mlsanity.com/paper/2303.05699/) (2023)
- [ChatFace: Chat-Guided Real Face Editing via Diffusion Latent Space Manipulation](https://mlsanity.com/paper/2305.14742/) (2023)
- [LinkGAN: Linking GAN Latents to Pixels for Controllable Image Synthesis](https://mlsanity.com/paper/2301.04604/) (2023)
- [Bounds all around: training energy-based models with bidirectional bounds](https://mlsanity.com/paper/2111.00929/) (2021)
- [NaviNeRF: NeRF-based 3D Representation Disentanglement by Latent Semantic Navigation](https://mlsanity.com/paper/2304.11342/) (2023)
- [Cycle Encoding of a StyleGAN Encoder for Improved Reconstruction and Editability](https://mlsanity.com/paper/2207.09367/) (2022)
- [What Decreases Editing Capability? Domain-Specific Hybrid Refinement for Improved GAN Inversion](https://mlsanity.com/paper/2301.12141/) (2023)
- [FaceDNeRF: Semantics-Driven Face Reconstruction, Prompt Editing and Relighting with Diffusion Models](https://mlsanity.com/paper/2306.00783/) (2023)
- [What's Inside Your Diffusion Model? A Score-Based Riemannian Metric to Explore the Data Manifold](https://mlsanity.com/paper/2505.11128/) (2025)
- [Mario Plays on a Manifold: Generating Functional Content in Latent Space through Differential Geometry](https://mlsanity.com/paper/2206.00106/) (2022)
- [3D-Aware Encoding for Style-based Neural Radiance Fields](https://mlsanity.com/paper/2211.06583/) (2022)
- [High-performance Evolutionary Algorithms for Online Neuron Control](https://mlsanity.com/paper/2204.06765/) (2022)
- [LSAP: Rethinking Inversion Fidelity, Perception and Editability in GAN Latent Space](https://mlsanity.com/paper/2209.12746/) (2022)
- [Disentanglement Learning via Topology](https://mlsanity.com/paper/2308.12696/) (2023)
- [PaQ-DETR: Learning Pattern and Quality-Aware Dynamic Queries for Object Detection](https://mlsanity.com/paper/2603.06917/) (2026)
- [A Sample-Level Evaluation and Generative Framework for Model Inversion Attacks](https://mlsanity.com/paper/2502.19070/) (2025)
- [SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Editing](https://mlsanity.com/paper/2112.00180/) (2021)
- [Which Style Makes Me Attractive? Interpretable Control Discovery and Counterfactual Explanation on StyleGAN](https://mlsanity.com/paper/2201.09689/) (2022)
- [Learning Input-agnostic Manipulation Directions in StyleGAN with Text Guidance](https://mlsanity.com/paper/2302.13331/) (2023)
- [StylePart: Image-based Shape Part Manipulation](https://mlsanity.com/paper/2111.10520/) (2021)
- [Rethinking the editing of generative adversarial networks: a method to estimate editing vectors based on dimension reduction](https://mlsanity.com/paper/2305.09454/) (2023)
- [Spatial-Contextual Discrepancy Information Compensation for GAN Inversion](https://mlsanity.com/paper/2312.07079/) (2023)
- [Exploring Biologically Inspired Mechanisms of Adversarial Robustness](https://mlsanity.com/paper/2405.00679/) (2024)

## Abstract (arXiv)

Generative adversarial networks (GANs) have emerged as a powerful unsupervised method to model the statistical patterns of real-world data sets, such as natural images. These networks are trained to map random inputs in their latent space to new samples representative of the learned data. However, the structure of the latent space is hard to intuit due to its high dimensionality and the non-linearity of the generator, which limits the usefulness of the models. Understanding the latent space requires a way to identify input codes for existing real-world images (inversion), and a way to identify directions with known image transformations (interpretability). Here, we use a geometric framework to address both issues simultaneously. We develop an architecture-agnostic method to compute the Riemannian metric of the image manifold created by GANs. The eigen-decomposition of the metric isolates axes that account for different levels of image variability. An empirical analysis of several pretrained GANs shows that image variation around each position is concentrated along surprisingly few major axes (the space is highly anisotropic) and the directions that create this large variation are similar at different positions in the space (the space is homogeneous). We show that many of the top eigenvectors correspond to interpretable transforms in the image space, with a substantial part of eigenspace corresponding to minor transforms which could be compressed out. This geometric understanding unifies key previous results related to GAN interpretability. We show that the use of this metric allows for more efficient optimization in the latent space (e.g. GAN inversion) and facilitates unsupervised discovery of interpretable axes. Our results illustrate that defining the geometry of the GAN image manifold can serve as a general framework for understanding GANs.