# Heterogeneous manifolds for curvature-aware graph embedding

arXiv: https://arxiv.org/abs/2202.01185 · HTML: https://mlsanity.com/paper/2202.01185/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 68 references that are in the map.
- First posted on arXiv 2 Feb 2022.

## Builds on (most cited first, 25 of 25)

- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Understanding over-squashing and bottlenecks on graphs via curvature](https://mlsanity.com/paper/2111.14522/) (2021)
- [Hyperbolic Graph Convolutional Neural Networks](https://mlsanity.com/paper/1910.12933/) (2019)
- [Hyperbolic Graph Neural Networks](https://mlsanity.com/paper/1910.12892/) (2019)
- [Fast Computation of Wasserstein Barycenters](https://mlsanity.com/paper/1310.4375/) (2013)
- [Representation Tradeoffs for Hyperbolic Embeddings](https://mlsanity.com/paper/1804.03329/) (2018)
- [Riemannian Adaptive Optimization Methods](https://mlsanity.com/paper/1810.00760/) (2018)
- [Networks beyond pairwise interactions: structure and dynamics](https://mlsanity.com/paper/2006.01764/) (2020)
- [Hyperbolic Attention Networks](https://mlsanity.com/paper/1805.09786/) (2018)
- [Higher-order organization of complex networks](https://mlsanity.com/paper/1612.08447/) (2016)
- [Constant Curvature Graph Convolutional Networks](https://mlsanity.com/paper/1911.05076/) (2019)
- [Mixed-curvature Variational Autoencoders](https://mlsanity.com/paper/1911.08411/) (2019)
- [Beltrami Flow and Neural Diffusion on Graphs](https://mlsanity.com/paper/2110.09443/) (2021)
- [Community Detection on Networks with Ricci Flow](https://mlsanity.com/paper/1907.03993/) (2019)
- [Building Deep Networks on Grassmann Manifolds](https://mlsanity.com/paper/1611.05742/) (2016)
- [Neural Embeddings of Graphs in Hyperbolic Space](https://mlsanity.com/paper/1705.10359/) (2017)
- [Computationally Tractable Riemannian Manifolds for Graph Embeddings](https://mlsanity.com/paper/2002.08665/) (2020)
- [Sinkhorn Barycenters with Free Support via Frank-Wolfe Algorithm](https://mlsanity.com/paper/1905.13194/) (2019)
- [Symmetric Spaces for Graph Embeddings: A Finsler-Riemannian Approach](https://mlsanity.com/paper/2106.04941/) (2021)
- [Vector-valued Distance and Gyrocalculus on the Space of Symmetric Positive Definite Matrices](https://mlsanity.com/paper/2110.13475/) (2021)
- [Gradient descent in hyperbolic space](https://mlsanity.com/paper/1805.08207/) (2018)
- [Directed Graph Embeddings in Pseudo-Riemannian Manifolds](https://mlsanity.com/paper/2106.08678/) (2021)

## Built on by (most cited first, 23 of 23)

- [On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topology](https://mlsanity.com/paper/2302.02941/) (2023)
- [Latent Graph Inference using Product Manifolds](https://mlsanity.com/paper/2211.16199/) (2022)
- [Motif-aware Riemannian Graph Neural Network with Generative-Contrastive Learning](https://mlsanity.com/paper/2401.01232/) (2024)
- [DeepRicci: Self-supervised Graph Structure-Feature Co-Refinement for Alleviating Over-squashing](https://mlsanity.com/paper/2401.12780/) (2024)
- [Contrastive Graph Clustering in Curvature Spaces](https://mlsanity.com/paper/2305.03555/) (2023)
- [Curve Your Attention: Mixed-Curvature Transformers for Graph Representation Learning](https://mlsanity.com/paper/2309.04082/) (2023)
- [Small Transformers Compute Universal Metric Embeddings](https://mlsanity.com/paper/2209.06788/) (2022)
- [Bringing motion taxonomies to continuous domains via GPLVM on hyperbolic manifolds](https://mlsanity.com/paper/2210.01672/) (2022)
- [Neural Snowflakes: Universal Latent Graph Inference via Trainable Latent Geometries](https://mlsanity.com/paper/2310.15003/) (2023)
- [Neural Spacetimes for DAG Representation Learning](https://mlsanity.com/paper/2408.13885/) (2024)
- [Improving Heterogeneous Graph Learning with Weighted Mixed-Curvature Product Manifold](https://mlsanity.com/paper/2307.04514/) (2023)
- [Capacity Bounds for Hyperbolic Neural Network Representations of Latent Tree Structures](https://mlsanity.com/paper/2308.09250/) (2023)
- [Product Manifold Representations for Learning on Biological Pathways](https://mlsanity.com/paper/2401.15478/) (2024)
- [Robust Hyperbolic Learning with Curvature-Aware Optimization](https://mlsanity.com/paper/2405.13979/) (2024)
- [Normed Spaces for Graph Embedding](https://mlsanity.com/paper/2312.01502/) (2023)
- [Current Symmetry Group Equivariant Convolution Frameworks for Representation Learning](https://mlsanity.com/paper/2409.07327/) (2024)
- [Mixed-curvature decision trees and random forests](https://mlsanity.com/paper/2410.13879/) (2024)
- [Over-Squashing in Riemannian Graph Neural Networks](https://mlsanity.com/paper/2311.15945/) (2023)
- [Improving embedding of graphs with missing data by soft manifolds](https://mlsanity.com/paper/2311.17598/) (2023)
- [Tuning the Geometry of Graph Neural Networks](https://mlsanity.com/paper/2207.05887/) (2022)
- [Topological Neural Networks: Mitigating the Bottlenecks of Graph Neural Networks via Higher-Order Interactions](https://mlsanity.com/paper/2402.06908/) (2024)
- [Mixed-Curvature Decision Trees and Random Forests](https://mlsanity.com/paper/2406.05227/) (2024)
- [Sobolev--Ricci Curvature](https://mlsanity.com/paper/2603.12652/) (2026)

## Abstract (arXiv)

Graph embeddings, wherein the nodes of the graph are represented by points in a continuous space, are used in a broad range of Graph ML applications. The quality of such embeddings crucially depends on whether the geometry of the space matches that of the graph. Euclidean spaces are often a poor choice for many types of real-world graphs, where hierarchical structure and a power-law degree distribution are linked to negative curvature. In this regard, it has recently been shown that hyperbolic spaces and more general manifolds, such as products of constant-curvature spaces and matrix manifolds, are advantageous to approximately match nodes pairwise distances. However, all these classes of manifolds are homogeneous, implying that the curvature distribution is the same at each point, making them unsuited to match the local curvature (and related structural properties) of the graph. In this paper, we study graph embeddings in a broader class of heterogeneous rotationally-symmetric manifolds. By adding a single extra radial dimension to any given existing homogeneous model, we can both account for heterogeneous curvature distributions on graphs and pairwise distances. We evaluate our approach on reconstruction tasks on synthetic and real datasets and show its potential in better preservation of high-order structures and heterogeneous random graphs generation.