# GeomNet: A Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Based Interaction Recognition

arXiv: https://arxiv.org/abs/2111.13089 · HTML: https://mlsanity.com/paper/2111.13089/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 68 references that are in the map.
- First posted on arXiv 25 Nov 2021.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/1904.12659/) (2019)
- [Riemannian Adaptive Optimization Methods](https://mlsanity.com/paper/1810.00760/) (2018)
- [A Riemannian Network for SPD Matrix Learning](https://mlsanity.com/paper/1608.04233/) (2016)
- [An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1902.09130/) (2019)
- [A New Representation of Skeleton Sequences for 3D Action Recognition](https://mlsanity.com/paper/1703.03492/) (2017)
- [Riemannian batch normalization for SPD neural networks](https://mlsanity.com/paper/1909.02414/) (2019)
- [geomstats: a Python Package for Riemannian Geometry in Machine Learning](https://mlsanity.com/paper/1805.08308/) (2018)
- [Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networks](https://mlsanity.com/paper/1603.07772/) (2016)
- [Building Deep Networks on Grassmann Manifolds](https://mlsanity.com/paper/1611.05742/) (2016)
- [Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates](https://mlsanity.com/paper/1706.08276/) (2017)
- [Modeling Temporal Dynamics and Spatial Configurations of Actions Using Two-Stream Recurrent Neural Networks](https://mlsanity.com/paper/1704.02581/) (2017)
- [A Survey on 3D Skeleton-Based Action Recognition Using Learning Method](https://mlsanity.com/paper/2002.05907/) (2020)
- [A neural network based on SPD manifold learning for skeleton-based hand gesture recognition](https://mlsanity.com/paper/1904.12970/) (2019)
- [Is Second-order Information Helpful for Large-scale Visual Recognition?](https://mlsanity.com/paper/1703.08050/) (2017)
- [Deep Learning on Lie Groups for Skeleton-based Action Recognition](https://mlsanity.com/paper/1612.05877/) (2016)
- [Interaction Relational Network for Mutual Action Recognition](https://mlsanity.com/paper/1910.04963/) (2019)
- [Parallel Transport on the Cone Manifold of SPD Matrices for Domain Adaptation](https://mlsanity.com/paper/1807.10479/) (2018)
- [Spatio-Temporal Graph Convolution for Skeleton Based Action Recognition](https://mlsanity.com/paper/1802.09834/) (2018)
- [Statistics on the (compact) Stiefel manifold: Theory and Applications](https://mlsanity.com/paper/1708.00045/) (2017)

## Built on by (most cited first, 27 of 27)

- [Riemannian Local Mechanism for SPD Neural Networks](https://mlsanity.com/paper/2201.10145/) (2022)
- [COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality](https://mlsanity.com/paper/2112.05892/) (2021)
- [Building Neural Networks on Matrix Manifolds: A Gyrovector Space Approach](https://mlsanity.com/paper/2305.04560/) (2023)
- [SPD-DDPM: Denoising Diffusion Probabilistic Models in the Symmetric Positive Definite Space](https://mlsanity.com/paper/2312.08200/) (2023)
- [Adaptive Log-Euclidean Metrics for SPD Matrix Learning](https://mlsanity.com/paper/2303.15477/) (2023)
- [Matrix Manifold Neural Networks++](https://mlsanity.com/paper/2405.19206/) (2024)
- [A numerical approximation method for the Fisher-Rao distance between multivariate normal distributions](https://mlsanity.com/paper/2302.08175/) (2023)
- [Riemannian Multinomial Logistics Regression for SPD Neural Networks](https://mlsanity.com/paper/2305.11288/) (2023)
- [ERA: Expert Retrieval and Assembly for Early Action Prediction](https://mlsanity.com/paper/2207.09675/) (2022)
- [Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry](https://mlsanity.com/paper/2407.10484/) (2024)
- [RMLR: Extending Multinomial Logistic Regression into General Geometries](https://mlsanity.com/paper/2409.19433/) (2024)
- [Two-person Graph Convolutional Network for Skeleton-based Human Interaction Recognition](https://mlsanity.com/paper/2208.06174/) (2022)
- [Learning to Normalize on the SPD Manifold under Bures-Wasserstein Geometry](https://mlsanity.com/paper/2504.00660/) (2025)
- [Neural Networks on Symmetric Spaces of Noncompact Type](https://mlsanity.com/paper/2601.01097/) (2026)
- [Video-based Contrastive Learning on Decision Trees: from Action Recognition to Autism Diagnosis](https://mlsanity.com/paper/2304.10073/) (2023)
- [Wrapped Gaussian on the manifold of Symmetric Positive Definite Matrices](https://mlsanity.com/paper/2502.01512/) (2025)
- [SMLNet: A SPD Manifold Learning Network for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2411.10679/) (2024)
- [Efficient and Accurate Skeleton-Based Two-Person Interaction Recognition Using Inter- and Intra-body Graphs](https://mlsanity.com/paper/2207.12648/) (2022)
- [DFNN: A Deep Fr'echet Neural Network Framework for Learning Metric-Space-Valued Responses](https://mlsanity.com/paper/2510.17072/) (2025)
- [Cross-view Action Recognition via Contrastive View-invariant Representation](https://mlsanity.com/paper/2305.01733/) (2023)
- [SkeleTR: Towrads Skeleton-based Action Recognition in the Wild](https://mlsanity.com/paper/2309.11445/) (2023)
- [Learning Adaptive Node Selection with External Attention for Human Interaction Recognition](https://mlsanity.com/paper/2507.03936/) (2025)
- [Siegel Neural Networks](https://mlsanity.com/paper/2511.09577/) (2025)
- [Two-stream Multi-level Dynamic Point Transformer for Two-person Interaction Recognition](https://mlsanity.com/paper/2307.11973/) (2023)
- [An Improved Graph Pooling Network for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2404.16359/) (2024)
- [E2E-GNet: An End-to-End Skeleton-based Geometric Deep Neural Network for Human Motion Recognition](https://mlsanity.com/paper/2603.02477/) (2026)
- [ReManNet: A Riemannian Manifold Network for Monocular 3D Lane Detection](https://mlsanity.com/paper/2603.19776/) (2026)

## Abstract (arXiv)

In this paper, we propose a novel method for representation and classification of two-person interactions from 3D skeleton sequences. The key idea of our approach is to use Gaussian distributions to capture statistics on R n and those on the space of symmetric positive definite (SPD) matrices. The main challenge is how to parametrize those distributions. Towards this end, we develop methods for embedding Gaussian distributions in matrix groups based on the theory of Lie groups and Riemannian symmetric spaces. Our method relies on the Riemannian geometry of the underlying manifolds and has the advantage of encoding high-order statistics from 3D joint positions. We show that the proposed method achieves competitive results in two-person interaction recognition on three benchmarks for 3D human activity understanding.