# A neural network based on SPD manifold learning for skeleton-based hand gesture recognition

arXiv: https://arxiv.org/abs/1904.12970 · HTML: https://mlsanity.com/paper/1904.12970/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2019.

## Built on by (most cited first, 30 of 46)

- [GeomNet: A Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Based Interaction Recognition](https://mlsanity.com/paper/2111.13089/) (2021)
- [Transductive Few-Shot Classification on the Oblique Manifold](https://mlsanity.com/paper/2108.04009/) (2021)
- [Vector-valued Distance and Gyrocalculus on the Space of Symmetric Positive Definite Matrices](https://mlsanity.com/paper/2110.13475/) (2021)
- [Building Neural Networks on Matrix Manifolds: A Gyrovector Space Approach](https://mlsanity.com/paper/2305.04560/) (2023)
- [SPD-DDPM: Denoising Diffusion Probabilistic Models in the Symmetric Positive Definite Space](https://mlsanity.com/paper/2312.08200/) (2023)
- [Decoupling and Recoupling Spatiotemporal Representation for RGB-D-based Motion Recognition](https://mlsanity.com/paper/2112.09129/) (2021)
- [Matrix Manifold Neural Networks++](https://mlsanity.com/paper/2405.19206/) (2024)
- [Multi-manifold Attention for Vision Transformers](https://mlsanity.com/paper/2207.08569/) (2022)
- [In My Perspective, In My Hands: Accurate Egocentric 2D Hand Pose and Action Recognition](https://mlsanity.com/paper/2404.09308/) (2024)
- [Riemannian Multinomial Logistics Regression for SPD Neural Networks](https://mlsanity.com/paper/2305.11288/) (2023)
- [A Two-stream Neural Network for Pose-based Hand Gesture Recognition](https://mlsanity.com/paper/2101.08926/) (2021)
- [Deep Efficient Continuous Manifold Learning for Time Series Modeling](https://mlsanity.com/paper/2112.03379/) (2021)
- [Modeling Graphs Beyond Hyperbolic: Graph Neural Networks in Symmetric Positive Definite Matrices](https://mlsanity.com/paper/2306.14064/) (2023)
- [HAN: An Efficient Hierarchical Self-Attention Network for Skeleton-Based Gesture Recognition](https://mlsanity.com/paper/2106.13391/) (2021)
- [Ultra-Range Gesture Recognition using a Web-Camera in Human-Robot Interaction](https://mlsanity.com/paper/2311.15361/) (2023)
- [Neural Networks on Symmetric Spaces of Noncompact Type](https://mlsanity.com/paper/2601.01097/) (2026)
- [Analyzing Deep Transformer Models for Time Series Forecasting via Manifold Learning](https://mlsanity.com/paper/2410.13792/) (2024)
- [Riemannian Self-Attention Mechanism for SPD Networks](https://mlsanity.com/paper/2311.16738/) (2023)
- [Schur's Positive-Definite Network: Deep Learning in the SPD cone with structure](https://mlsanity.com/paper/2406.09023/) (2024)
- [OCHID-Fi: Occlusion-Robust Hand Pose Estimation in 3D via RF-Vision](https://mlsanity.com/paper/2308.10146/) (2023)
- [Real-Time Hand Gesture Recognition: Integrating Skeleton-Based Data Fusion and Multi-Stream CNN](https://mlsanity.com/paper/2406.15003/) (2024)
- [Action Recognition in Real-World Ambient Assisted Living Environment](https://mlsanity.com/paper/2503.23214/) (2025)
- [Data-Free Class-Incremental Gesture Recognition with Prototype-Guided Pseudo Feature Replay](https://mlsanity.com/paper/2505.20049/) (2025)
- [DFNN: A Deep Fr'echet Neural Network Framework for Learning Metric-Space-Valued Responses](https://mlsanity.com/paper/2510.17072/) (2025)
- [Fast Monocular Hand Pose Estimation on Embedded Systems](https://mlsanity.com/paper/2102.07067/) (2021)
- [Fine-grained Hand Gesture Recognition in Multi-viewpoint Hand Hygiene](https://mlsanity.com/paper/2109.02917/) (2021)
- [DreamNet: A Deep Riemannian Network based on SPD Manifold Learning for Visual Classification](https://mlsanity.com/paper/2206.07967/) (2022)
- [Rieoptax: Riemannian Optimization in JAX](https://mlsanity.com/paper/2210.04840/) (2022)
- [mSPD-NN: A Geometrically Aware Neural Framework for Biomarker Discovery from Functional Connectomics Manifolds](https://mlsanity.com/paper/2303.14986/) (2023)
- [Boosting Gesture Recognition with an Automatic Gesture Annotation Framework](https://mlsanity.com/paper/2401.11150/) (2024)

## Abstract (arXiv)

This paper proposes a new neural network based on SPD manifold learning for skeleton-based hand gesture recognition. Given the stream of hand's joint positions, our approach combines two aggregation processes on respectively spatial and temporal domains. The pipeline of our network architecture consists in three main stages. The first stage is based on a convolutional layer to increase the discriminative power of learned features. The second stage relies on different architectures for spatial and temporal Gaussian aggregation of joint features. The third stage learns a final SPD matrix from skeletal data. A new type of layer is proposed for the third stage, based on a variant of stochastic gradient descent on Stiefel manifolds. The proposed network is validated on two challenging datasets and shows state-of-the-art accuracies on both datasets.