# Expressivity of congruence-based architectures for DNNs on positive-definite matrices

arXiv: https://arxiv.org/abs/2606.02490 · HTML: https://mlsanity.com/paper/2606.02490/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 1 Jun 2026.

## Builds on (most cited first, 13 of 13)

- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Implicit Regularization in Deep Matrix Factorization](https://mlsanity.com/paper/1905.13655/) (2019)
- [On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization](https://mlsanity.com/paper/1802.06509/) (2018)
- [A Riemannian Network for SPD Matrix Learning](https://mlsanity.com/paper/1608.04233/) (2016)
- [Activation Functions: Comparison of trends in Practice and Research for Deep Learning](https://mlsanity.com/paper/1811.03378/) (2018)
- [Riemannian batch normalization for SPD neural networks](https://mlsanity.com/paper/1909.02414/) (2019)
- [Dimensionality Reduction on SPD Manifolds: The Emergence of Geometry-Aware Methods](https://mlsanity.com/paper/1605.06182/) (2016)
- [Vector-valued Distance and Gyrocalculus on the Space of Symmetric Positive Definite Matrices](https://mlsanity.com/paper/2110.13475/) (2021)
- [Kernel Methods on the Riemannian Manifold of Symmetric Positive Definite Matrices](https://mlsanity.com/paper/1412.4172/) (2014)
- [Building Neural Networks on Matrix Manifolds: A Gyrovector Space Approach](https://mlsanity.com/paper/2305.04560/) (2023)
- [Matrix Manifold Neural Networks++](https://mlsanity.com/paper/2405.19206/) (2024)
- [Deep Riemannian Networks for End-to-End EEG Decoding](https://mlsanity.com/paper/2212.10426/) (2022)
- [Universal kernels via harmonic analysis on Riemannian symmetric spaces](https://mlsanity.com/paper/2506.19245/) (2025)

## Abstract (arXiv)

This work studies neural architectures for classifying symmetric positive-definite matrices, focusing on congruence-like layers, in which the input matrix is multiplied on the left and right by a (possibly rectangular) weight matrix W and its transpose. Such layers lie at the core of the celebrated SPDNet and have also been employed independently for dimensionality reduction on positive-definite data. We show that the (semi)-orthogonality constraint commonly imposed on W limits the expressivity of these layers: for certain activation functions, the resulting architecture collapses to a one-hidden-layer equivalent. This lack of expressivity follows from a loss of spectral diversity in congruence-like layers for semi-orthogonal W and is a direct consequence of Poincar'e's separation theorem. We then examine the choice of the final classifier, comparing several Riemannian classifiers and discussing their compatibility with the feature maps produced by congruence-like layers.