# Building Deep Networks on Grassmann Manifolds

arXiv: https://arxiv.org/abs/1611.05742 · HTML: https://mlsanity.com/paper/1611.05742/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2016.

## Built on by (most cited first, 30 of 58)

- [GeomNet: A Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Based Interaction Recognition](https://mlsanity.com/paper/2111.13089/) (2021)
- [Symmetric Spaces for Graph Embeddings: A Finsler-Riemannian Approach](https://mlsanity.com/paper/2106.04941/) (2021)
- [Heterogeneous manifolds for curvature-aware graph embedding](https://mlsanity.com/paper/2202.01185/) (2022)
- [Riemannian Local Mechanism for SPD Neural Networks](https://mlsanity.com/paper/2201.10145/) (2022)
- [A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold](https://mlsanity.com/paper/2302.08210/) (2023)
- [Building Neural Networks on Matrix Manifolds: A Gyrovector Space Approach](https://mlsanity.com/paper/2305.04560/) (2023)
- [Matrix Manifold Neural Networks++](https://mlsanity.com/paper/2405.19206/) (2024)
- [A Two-stream Neural Network for Pose-based Hand Gesture Recognition](https://mlsanity.com/paper/2101.08926/) (2021)
- [O-ViT: Orthogonal Vision Transformer](https://mlsanity.com/paper/2201.12133/) (2022)
- [The Manifold Scattering Transform for High-Dimensional Point Cloud Data](https://mlsanity.com/paper/2206.10078/) (2022)
- [A Lie Group Approach to Riemannian Batch Normalization](https://mlsanity.com/paper/2403.11261/) (2024)
- [RMLR: Extending Multinomial Logistic Regression into General Geometries](https://mlsanity.com/paper/2409.19433/) (2024)
- [HAN: An Efficient Hierarchical Self-Attention Network for Skeleton-Based Gesture Recognition](https://mlsanity.com/paper/2106.13391/) (2021)
- [Polynomial Chaos Expansions on Principal Geodesic Grassmannian Submanifolds for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/2401.16683/) (2024)
- [Neural Networks on Symmetric Spaces of Noncompact Type](https://mlsanity.com/paper/2601.01097/) (2026)
- [tensorflow-riemopt: A Library for Optimization on Riemannian Manifolds](https://mlsanity.com/paper/2105.13921/) (2021)
- [Intrinsic and extrinsic deep learning on manifolds](https://mlsanity.com/paper/2302.08606/) (2023)
- [Riemannian Self-Attention Mechanism for SPD Networks](https://mlsanity.com/paper/2311.16738/) (2023)
- [Grassmannian learning mutual subspace method for image set recognition](https://mlsanity.com/paper/2111.04352/) (2021)
- [Grassmann Manifold Flows for Stable Shape Generation](https://mlsanity.com/paper/2211.02900/) (2022)
- [Federated PCA on Grassmann Manifold for IoT Anomaly Detection](https://mlsanity.com/paper/2407.07421/) (2024)
- [Embedding Graphs on Grassmann Manifold](https://mlsanity.com/paper/2205.15068/) (2022)
- [Normed Spaces for Graph Embedding](https://mlsanity.com/paper/2312.01502/) (2023)
- [GrFormer: A Novel Transformer on Grassmann Manifold for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2506.14384/) (2025)
- [Think as Needed: Geometry-Driven Adaptive Perception for Autonomous Driving](https://mlsanity.com/paper/2605.10117/) (2026)
- [BrainRiem: Riemannian Prototype Learning for Source-Free Cross-Site Brain Network Diagnosis](https://mlsanity.com/paper/2606.29200/) (2026)
- [Skeleton-based Hand-Gesture Recognition with Lightweight Graph Convolutional Networks](https://mlsanity.com/paper/2104.04255/) (2021)
- [Revisiting 2D Convolutional Neural Networks for Graph-based Applications](https://mlsanity.com/paper/2105.11016/) (2021)
- [DreamNet: A Deep Riemannian Network based on SPD Manifold Learning for Visual Classification](https://mlsanity.com/paper/2206.07967/) (2022)
- [Rieoptax: Riemannian Optimization in JAX](https://mlsanity.com/paper/2210.04840/) (2022)

## Abstract (arXiv)

Learning representations on Grassmann manifolds is popular in quite a few visual recognition tasks. In order to enable deep learning on Grassmann manifolds, this paper proposes a deep network architecture by generalizing the Euclidean network paradigm to Grassmann manifolds. In particular, we design full rank mapping layers to transform input Grassmannian data to more desirable ones, exploit re-orthonormalization layers to normalize the resulting matrices, study projection pooling layers to reduce the model complexity in the Grassmannian context, and devise projection mapping layers to respect Grassmannian geometry and meanwhile achieve Euclidean forms for regular output layers. To train the Grassmann networks, we exploit a stochastic gradient descent setting on manifolds of the connection weights, and study a matrix generalization of backpropagation to update the structured data. The evaluations on three visual recognition tasks show that our Grassmann networks have clear advantages over existing Grassmann learning methods, and achieve results comparable with state-of-the-art approaches.