# An elementary introduction to information geometry

arXiv: https://arxiv.org/abs/1808.08271 · HTML: https://mlsanity.com/paper/1808.08271/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Aug 2018.

## Built on by (most cited first, 30 of 76)

- [Merging Models with Fisher-Weighted Averaging](https://mlsanity.com/paper/2111.09832/) (2021)
- [Fisher Flow Matching for Generative Modeling over Discrete Data](https://mlsanity.com/paper/2405.14664/) (2024)
- [Loss Minimization through the Lens of Outcome Indistinguishability](https://mlsanity.com/paper/2210.08649/) (2022)
- [All Roads Lead to Likelihood: The Value of Reinforcement Learning in Fine-Tuning](https://mlsanity.com/paper/2503.01067/) (2025)
- [Wasserstein Flow Matching: Generative modeling over families of distributions](https://mlsanity.com/paper/2411.00698/) (2024)
- [Improved Regret Bounds of (Multinomial) Logistic Bandits via Regret-to-Confidence-Set Conversion](https://mlsanity.com/paper/2310.18554/) (2023)
- [On the Properties of Kullback-Leibler Divergence Between Multivariate Gaussian Distributions](https://mlsanity.com/paper/2102.05485/) (2021)
- [Pulling back information geometry](https://mlsanity.com/paper/2106.05367/) (2021)
- [On the Variance of the Fisher Information for Deep Learning](https://mlsanity.com/paper/2107.04205/) (2021)
- [Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer](https://mlsanity.com/paper/2201.13094/) (2022)
- [Bayesian Renormalization](https://mlsanity.com/paper/2305.10491/) (2023)
- [Learning to Assist Humans without Inferring Rewards](https://mlsanity.com/paper/2411.02623/) (2024)
- [FIT: A Metric for Model Sensitivity](https://mlsanity.com/paper/2210.08502/) (2022)
- [The Information Geometry of Softmax: Probing and Steering](https://mlsanity.com/paper/2602.15293/) (2026)
- [Foundations of Top-k Decoding For Language Models](https://mlsanity.com/paper/2505.19371/) (2025)
- [Model-centric Data Manifold: the Data Through the Eyes of the Model](https://mlsanity.com/paper/2104.13289/) (2021)
- [A Class of Geometric Structures in Transfer Learning: Minimax Bounds and Optimality](https://mlsanity.com/paper/2202.11685/) (2022)
- [Bayesian RG Flow in Neural Network Field Theories](https://mlsanity.com/paper/2405.17538/) (2024)
- [Language Model Decoding as Direct Metrics Optimization](https://mlsanity.com/paper/2310.01041/) (2023)
- [NCoder -- A Quantum Field Theory approach to encoding data](https://mlsanity.com/paper/2402.00944/) (2024)
- [On the Importance of Firth Bias Reduction in Few-Shot Classification](https://mlsanity.com/paper/2110.02529/) (2021)
- [Adversarial attacks on neural networks through canonical Riemannian foliations](https://mlsanity.com/paper/2203.00922/) (2022)
- [Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents](https://mlsanity.com/paper/2203.10592/) (2022)
- [Phase2vec: Dynamical systems embedding with a physics-informed convolutional network](https://mlsanity.com/paper/2212.03857/) (2022)
- [The Fisher-Rao geometry of CES distributions](https://mlsanity.com/paper/2310.01032/) (2023)
- [Enforcing Latent Euclidean Geometry in Single-Cell VAEs for Manifold Interpolation](https://mlsanity.com/paper/2507.11789/) (2025)
- [AEM: Adaptive Entropy Modulation for Multi-Turn Agentic Reinforcement Learning](https://mlsanity.com/paper/2605.00425/) (2026)
- [A Unifying and Canonical Description of Measure-Preserving Diffusions](https://mlsanity.com/paper/2105.02845/) (2021)
- [Towards Optimal Compression: Joint Pruning and Quantization](https://mlsanity.com/paper/2302.07612/) (2023)
- [Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Attacks?](https://mlsanity.com/paper/2505.12871/) (2025)

## Abstract (arXiv)

In this survey, we describe the fundamental differential-geometric structures of information manifolds, state the fundamental theorem of information geometry, and illustrate some use cases of these information manifolds in information sciences. The exposition is self-contained by concisely introducing the necessary concepts of differential geometry, but proofs are omitted for brevity.