# Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry

arXiv: https://arxiv.org/abs/2407.10484 · HTML: https://mlsanity.com/paper/2407.10484/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 70 references that are in the map.
- First posted on arXiv 15 Jul 2024.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Hyperbolic Neural Networks](https://mlsanity.com/paper/1805.09112/) (2018)
- [Bilinear CNNs for Fine-grained Visual Recognition](https://mlsanity.com/paper/1504.07889/) (2015)
- [Compact Bilinear Pooling](https://mlsanity.com/paper/1511.06062/) (2015)
- [Towards Faster Training of Global Covariance Pooling Networks by Iterative Matrix Square Root Normalization](https://mlsanity.com/paper/1712.01034/) (2017)
- [Learning Deep Bilinear Transformation for Fine-grained Image Representation](https://mlsanity.com/paper/1911.03621/) (2019)
- [Is Second-order Information Helpful for Large-scale Visual Recognition?](https://mlsanity.com/paper/1703.08050/) (2017)
- [Low-rank Bilinear Pooling for Fine-Grained Classification](https://mlsanity.com/paper/1611.05109/) (2016)
- [Deep Learning on Lie Groups for Skeleton-based Action Recognition](https://mlsanity.com/paper/1612.05877/) (2016)
- [Deep CNNs Meet Global Covariance Pooling: Better Representation and Generalization](https://mlsanity.com/paper/1904.06836/) (2019)
- [Covariance Pooling For Facial Expression Recognition](https://mlsanity.com/paper/1805.04855/) (2018)
- [On Riemannian Optimization over Positive Definite Matrices with the Bures-Wasserstein Geometry](https://mlsanity.com/paper/2106.00286/) (2021)
- [Improved Bilinear Pooling with CNNs](https://mlsanity.com/paper/1707.06772/) (2017)
- [Tensor Representations for Action Recognition](https://mlsanity.com/paper/2012.14371/) (2020)
- [GeomNet: A Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Based Interaction Recognition](https://mlsanity.com/paper/2111.13089/) (2021)
- [Why Approximate Matrix Square Root Outperforms Accurate SVD in Global Covariance Pooling?](https://mlsanity.com/paper/2105.02498/) (2021)
- [Riemannian Local Mechanism for SPD Neural Networks](https://mlsanity.com/paper/2201.10145/) (2022)
- [Distribution Knowledge Embedding for Graph Pooling](https://mlsanity.com/paper/2109.14333/) (2021)
- [Building Neural Networks on Matrix Manifolds: A Gyrovector Space Approach](https://mlsanity.com/paper/2305.04560/) (2023)
- [Fast Differentiable Matrix Square Root and Inverse Square Root](https://mlsanity.com/paper/2201.12543/) (2022)
- [Fast Differentiable Matrix Square Root](https://mlsanity.com/paper/2201.08663/) (2022)
- [Adaptive Log-Euclidean Metrics for SPD Matrix Learning](https://mlsanity.com/paper/2303.15477/) (2023)
- [Second-order Democratic Aggregation](https://mlsanity.com/paper/1808.07503/) (2018)
- [Riemannian Multinomial Logistics Regression for SPD Neural Networks](https://mlsanity.com/paper/2305.11288/) (2023)
- [Learning with symmetric positive definite matrices via generalized Bures-Wasserstein geometry](https://mlsanity.com/paper/2110.10464/) (2021)

## Built on by (most cited first, 9 of 9)

- [RMLR: Extending Multinomial Logistic Regression into General Geometries](https://mlsanity.com/paper/2409.19433/) (2024)
- [Learning to Normalize on the SPD Manifold under Bures-Wasserstein Geometry](https://mlsanity.com/paper/2504.00660/) (2025)
- [Sheaf Neural Networks on SPD Manifolds: Second-Order Geometric Representation Learning](https://mlsanity.com/paper/2604.20308/) (2026)
- [Shallow Neural Networks Learn Low-Degree Spherical Polynomials with Feature Learning by Learnable Channel Attention](https://mlsanity.com/paper/2512.20562/) (2025)
- [Riemannian Networks over Full-Rank Correlation Matrices](https://mlsanity.com/paper/2605.19073/) (2026)
- [A Sliced-Wasserstein Framework on Correlation Matrices for EEG Decoding](https://mlsanity.com/paper/2606.06104/) (2026)
- [Towards Robust EEG Decoding Based on Riemannian Self-Attention](https://mlsanity.com/paper/2606.25456/) (2026)
- [LieBN: Batch Normalization over Lie Groups](https://mlsanity.com/paper/2607.08783/) (2026)
- [Orthogonal Polynomial Approximation for Matrix Log Normalization in Global Covariance Pooling](https://mlsanity.com/paper/2608.19021/) (2026)

## Abstract (arXiv)

Global Covariance Pooling (GCP) has been demonstrated to improve the performance of Deep Neural Networks (DNNs) by exploiting second-order statistics of high-level representations. GCP typically performs classification of the covariance matrices by applying matrix function normalization, such as matrix logarithm or power, followed by a Euclidean classifier. However, covariance matrices inherently lie in a Riemannian manifold, known as the Symmetric Positive Definite (SPD) manifold. The current literature does not provide a satisfactory explanation of why Euclidean classifiers can be applied directly to Riemannian features after the normalization of the matrix power. To mitigate this gap, this paper provides a comprehensive and unified understanding of the matrix logarithm and power from a Riemannian geometry perspective. The underlying mechanism of matrix functions in GCP is interpreted from two perspectives: one based on tangent classifiers (Euclidean classifiers on the tangent space) and the other based on Riemannian classifiers. Via theoretical analysis and empirical validation through extensive experiments on fine-grained and large-scale visual classification datasets, we conclude that the working mechanism of the matrix functions should be attributed to the Riemannian classifiers they implicitly respect. The code is available at https://github.com/GitZH-Chen/RiemGCP.git.