mlsanityData to 7 Oct 2026

Pulling back information geometry

arXiv 2106.05367 ↗First posted 9 Jun 2021cs.LG stat.MLcited by 23

Not on the map. Why?

Abstract (arXiv)

Latent space geometry has shown itself to provide a rich and rigorous framework for interacting with the latent variables of deep generative models. The existing theory, however, relies on the decoder being a Gaussian distribution as its simple reparametrization allows us to interpret the generating process as a random projection of a deterministic manifold. Consequently, this approach breaks down when applied to decoders that are not as easily reparametrized. We here propose to use the Fisher-Rao metric associated with the space of decoder distributions as a reference metric, which we pull back to the latent space. We show that we can achieve meaningful latent geometries for a wide range of decoder distributions for which the previous theory was not applicable, opening the door to `black box' latent geometries.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 21 of its 42 references that are in the map. First posted on arXiv 9 Jun 2021.

Builds on

21 of 42 references.

References · 21

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  4. 2014New insights and perspectives on the natural gradient methodcited by 435
  5. 2018Pyro: Deep Universal Probabilistic Programmingcited by 379
  6. 2013Black Box Variational Inferencecited by 317
  7. 2018Hyperspherical Variational Auto-Encoderscited by 188
  8. 2017Latent Space Oddity: on the Curvature of Deep Generative Modelscited by 169
  9. 2017The Riemannian Geometry of Deep Generative Modelscited by 81
  10. 2018An elementary introduction to information geometrycited by 76
  11. 2019Reliable training and estimation of variance networkscited by 63
  12. 2017Metrics for Deep Generative Modelscited by 53
9 more
  1. 2020Geometrically Enriched Latent Spacescited by 45
  2. 2014Metrics for Probabilistic Geometriescited by 28
  3. 2019Fast and Robust Shortest Paths on Manifolds Learned from Datacited by 25
  4. 2016A Locally Adaptive Normal Distributioncited by 23
  5. 2021Learning Riemannian Manifolds for Geodesic Motion Skillscited by 23
  6. 2018Geodesic Clustering in Deep Generative Modelscited by 14
  7. 2013Probabilistic Solutions to Differential Equations and their Application to Riemannian Statisticscited by 11
  8. 2019Expected path length on random manifoldscited by 9
  9. 2018Active Learning based on Data Uncertainty and Model Sensitivitycited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 4; 2023: 0; 2024: 3; 2025: 8; 2026: 7.

23 papers.

Sort:

By area:

  1. 2024Metric Flow Matching for Smooth Interpolations on the Data Manifoldcited by 74
  2. 2021Detecting Model Misspecification in Amortized Bayesian Inference with Neural Networkscited by 48
  3. 2022Hyperbolic VAE via Latent Gaussian Distributionscited by 14
  4. 2026The Information Geometry of Softmax: Probing and Steeringcited by 9
  5. 2025Follow the Energy, Find the Path: Riemannian Metrics from Energy-Based Modelscited by 7
  6. 2025Riemannian-Geometric Fingerprints of Generative Modelscited by 7
  7. 2024Decoder ensembling for learned latent geometriescited by 6
  8. 2025The Spacetime of Diffusion Models: An Information Geometry Perspectivecited by 6
  9. 2022Mario Plays on a Manifold: Generating Functional Content in Latent Space through Differential Geometrycited by 5
  10. 2025Enforcing Latent Euclidean Geometry in Single-Cell VAEs for Manifold Interpolationcited by 5
  11. 2025Connecting Neural Models Latent Geometries with Relative Geodesic Representationscited by 4
  12. 2026Jacobian Scopes: token-level causal attributions in LLMscited by 4
11 more
  1. 2022Identifying latent distances with Finslerian geometrycited by 2
  2. 2025Geodesic Calculus on Implicitly Defined Latent Manifoldscited by 2
  3. 2026GeodesicNVS: Probability Density Geodesic Flow Matching for Novel View Synthesiscited by 2
  4. 2022The Fisher-Rao Loss for Learning under Label Noisecited by 1
  5. 2024Bayesian Unsupervised Disentanglement of Anatomy and Geometry for Deep Groupwise Image Registrationcited by 1
  6. 2025Supervised Quadratic Feature Analysis: Information Geometry Approach for Dimensionality Reductioncited by 0
  7. 2025A Likely Geometry of Generative Modelscited by 0
  8. 2026Hyperparameter Trajectory Inference with Conditional Lagrangian Optimal Transportcited by 0
  9. 2026A Theory of Speciation in Generative Diffusion Models on Compact Riemannian Manifoldscited by 0
  10. 2026Bayesian Optimization with Fisher Information Geometry: Gradient Bounds and Trust-Region Methodscited by 0
  11. 2026Fisher-IRG: Fisher-Induced Local Invariant Representation Geometry across Language and Vision Modelscited by 0