# Pulling back information geometry

arXiv: https://arxiv.org/abs/2106.05367 · HTML: https://mlsanity.com/paper/2106.05367/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 42 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 21 of 21)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [Hyperspherical Variational Auto-Encoders](https://mlsanity.com/paper/1804.00891/) (2018)
- [Latent Space Oddity: on the Curvature of Deep Generative Models](https://mlsanity.com/paper/1710.11379/) (2017)
- [The Riemannian Geometry of Deep Generative Models](https://mlsanity.com/paper/1711.08014/) (2017)
- [An elementary introduction to information geometry](https://mlsanity.com/paper/1808.08271/) (2018)
- [Reliable training and estimation of variance networks](https://mlsanity.com/paper/1906.03260/) (2019)
- [Metrics for Deep Generative Models](https://mlsanity.com/paper/1711.01204/) (2017)
- [Geometrically Enriched Latent Spaces](https://mlsanity.com/paper/2008.00565/) (2020)
- [Metrics for Probabilistic Geometries](https://mlsanity.com/paper/1411.7432/) (2014)
- [Fast and Robust Shortest Paths on Manifolds Learned from Data](https://mlsanity.com/paper/1901.07229/) (2019)
- [A Locally Adaptive Normal Distribution](https://mlsanity.com/paper/1606.02518/) (2016)
- [Learning Riemannian Manifolds for Geodesic Motion Skills](https://mlsanity.com/paper/2106.04315/) (2021)
- [Geodesic Clustering in Deep Generative Models](https://mlsanity.com/paper/1809.04747/) (2018)
- [Probabilistic Solutions to Differential Equations and their Application to Riemannian Statistics](https://mlsanity.com/paper/1306.0308/) (2013)
- [Expected path length on random manifolds](https://mlsanity.com/paper/1908.07377/) (2019)
- [Active Learning based on Data Uncertainty and Model Sensitivity](https://mlsanity.com/paper/1808.02026/) (2018)

## Built on by (most cited first, 23 of 23)

- [Metric Flow Matching for Smooth Interpolations on the Data Manifold](https://mlsanity.com/paper/2405.14780/) (2024)
- [Detecting Model Misspecification in Amortized Bayesian Inference with Neural Networks](https://mlsanity.com/paper/2112.08866/) (2021)
- [Hyperbolic VAE via Latent Gaussian Distributions](https://mlsanity.com/paper/2209.15217/) (2022)
- [The Information Geometry of Softmax: Probing and Steering](https://mlsanity.com/paper/2602.15293/) (2026)
- [Riemannian-Geometric Fingerprints of Generative Models](https://mlsanity.com/paper/2506.22802/) (2025)
- [Follow the Energy, Find the Path: Riemannian Metrics from Energy-Based Models](https://mlsanity.com/paper/2505.18230/) (2025)
- [Decoder ensembling for learned latent geometries](https://mlsanity.com/paper/2408.07507/) (2024)
- [The Spacetime of Diffusion Models: An Information Geometry Perspective](https://mlsanity.com/paper/2505.17517/) (2025)
- [Mario Plays on a Manifold: Generating Functional Content in Latent Space through Differential Geometry](https://mlsanity.com/paper/2206.00106/) (2022)
- [Enforcing Latent Euclidean Geometry in Single-Cell VAEs for Manifold Interpolation](https://mlsanity.com/paper/2507.11789/) (2025)
- [Connecting Neural Models Latent Geometries with Relative Geodesic Representations](https://mlsanity.com/paper/2506.01599/) (2025)
- [Jacobian Scopes: token-level causal attributions in LLMs](https://mlsanity.com/paper/2601.16407/) (2026)
- [Identifying latent distances with Finslerian geometry](https://mlsanity.com/paper/2212.10010/) (2022)
- [Geodesic Calculus on Implicitly Defined Latent Manifolds](https://mlsanity.com/paper/2510.09468/) (2025)
- [GeodesicNVS: Probability Density Geodesic Flow Matching for Novel View Synthesis](https://mlsanity.com/paper/2603.01010/) (2026)
- [The Fisher-Rao Loss for Learning under Label Noise](https://mlsanity.com/paper/2210.16401/) (2022)
- [Bayesian Unsupervised Disentanglement of Anatomy and Geometry for Deep Groupwise Image Registration](https://mlsanity.com/paper/2401.02141/) (2024)
- [A Theory of Speciation in Generative Diffusion Models on Compact Riemannian Manifolds](https://mlsanity.com/paper/2608.23798/) (2026)
- [Supervised Quadratic Feature Analysis: Information Geometry Approach for Dimensionality Reduction](https://mlsanity.com/paper/2502.00168/) (2025)
- [A Likely Geometry of Generative Models](https://mlsanity.com/paper/2510.26266/) (2025)
- [Hyperparameter Trajectory Inference with Conditional Lagrangian Optimal Transport](https://mlsanity.com/paper/2603.01771/) (2026)
- [Bayesian Optimization with Fisher Information Geometry: Gradient Bounds and Trust-Region Methods](https://mlsanity.com/paper/2609.31107/) (2026)
- [Fisher-IRG: Fisher-Induced Local Invariant Representation Geometry across Language and Vision Models](https://mlsanity.com/paper/2609.36458/) (2026)

## Abstract (arXiv)

Latent space geometry has shown itself to provide a rich and rigorous framework for interacting with the latent variables of deep generative models. The existing theory, however, relies on the decoder being a Gaussian distribution as its simple reparametrization allows us to interpret the generating process as a random projection of a deterministic manifold. Consequently, this approach breaks down when applied to decoders that are not as easily reparametrized. We here propose to use the Fisher-Rao metric associated with the space of decoder distributions as a reference metric, which we pull back to the latent space. We show that we can achieve meaningful latent geometries for a wide range of decoder distributions for which the previous theory was not applicable, opening the door to `black box' latent geometries.