# Beyond Unimodal Bases: Pullback Geometry for Multimodal Data

arXiv: https://arxiv.org/abs/2610.00708 · HTML: https://mlsanity.com/paper/2610.00708/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 18 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Builds on (most cited first, 12 of 12)

- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders](https://mlsanity.com/paper/1611.02648/) (2016)
- [Latent Space Oddity: on the Curvature of Deep Generative Models](https://mlsanity.com/paper/1710.11379/) (2017)
- [The Riemannian Geometry of Deep Generative Models](https://mlsanity.com/paper/1711.08014/) (2017)
- [Semi-Supervised Learning with Normalizing Flows](https://mlsanity.com/paper/1912.13025/) (2019)
- [Pulling back symmetric Riemannian geometry for data analysis](https://mlsanity.com/paper/2403.06612/) (2024)
- [GC-Flow: A Graph-Based Flow Network for Effective Clustering](https://mlsanity.com/paper/2305.17284/) (2023)
- [Manifold Learning with Normalizing Flows: Towards Regularity, Expressivity and Iso-Riemannian Geometry](https://mlsanity.com/paper/2505.08087/) (2025)
- [Iso-Riemannian Optimization on Learned Data Manifolds](https://mlsanity.com/paper/2510.21033/) (2025)
- [Learning Geometry and Topology via Multi-Chart Flows](https://mlsanity.com/paper/2505.24665/) (2025)
- [Riemannian Metric Matching for Scalable Geometric Modeling of Distributions](https://mlsanity.com/paper/2606.14334/) (2026)
- [Riemannian Archetypal Analysis: Interpretable non-linear data analysis on deformed star distributions](https://mlsanity.com/paper/2605.24113/) (2026)

## Abstract (arXiv)

Data-driven Riemannian geometry provides nonlinear interpolation and geometric representations of high-dimensional data. For these operations to be statistically meaningful, paths between observations should preferentially traverse high-likelihood regions. Existing scalable pullback constructions typically use a unimodal Gaussian latent distribution, assuming that the data reside close to a single manifold. For multimodal data, mapping separated modes or local structures into one Gaussian region can require substantial transport deformation and compromise the resulting geometry. We introduce a pullback geometry for data supported on mixtures of manifolds. Using a latent Gaussian mixture, we define its Riemannian metric as the matrix square of the responsibility-weighted expected component precision. The metric is smooth and positive definite and recovers the existing Gaussian construction in the single-component limit. For structured overlapping mixtures, we establish conditions under which the log-density is concave along geodesics, providing a formal connection between the proposed geometry and paths through high-likelihood regions, and derive the corresponding local curvature relations. We instantiate this geometry in a normalizing flow with adaptive mixture learning, allowing the number of active components to emerge from the data and supporting component-wise reconstruction and local effective-dimension estimation. Experiments on synthetic geometric data, a controlled multi-view image setting with a known reference trajectory, and MNIST show reduced transport distortion, competitive path support, close reference-trajectory recovery, and improved interpolation realism. These results extend scalable pullback geometry beyond datasets that reside close to a single manifold while retaining tractable and interpretable local structure.