# Spatio-Temporal Covariance Descriptors for Action and Gesture Recognition

arXiv: https://arxiv.org/abs/1303.6021 · HTML: https://mlsanity.com/paper/1303.6021/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Mar 2013.

## Built on by (most cited first, 4 of 4)

- [Temporal-attentive Covariance Pooling Networks for Video Recognition](https://mlsanity.com/paper/2110.14381/) (2021)
- [Riemannian Self-Attention Mechanism for SPD Networks](https://mlsanity.com/paper/2311.16738/) (2023)
- [Covariance Descriptors Meet General Vision Encoders: Riemannian Deep Learning for Medical Image Classification](https://mlsanity.com/paper/2511.04190/) (2025)
- [Towards Robust EEG Decoding Based on Riemannian Self-Attention](https://mlsanity.com/paper/2606.25456/) (2026)

## Abstract (arXiv)

We propose a new action and gesture recognition method based on spatio-temporal covariance descriptors and a weighted Riemannian locality preserving projection approach that takes into account the curved space formed by the descriptors. The weighted projection is then exploited during boosting to create a final multiclass classification algorithm that employs the most useful spatio-temporal regions. We also show how the descriptors can be computed quickly through the use of integral video representations. Experiments on the UCF sport, CK+ facial expression and Cambridge hand gesture datasets indicate superior performance of the proposed method compared to several recent state-of-the-art techniques. The proposed method is robust and does not require additional processing of the videos, such as foreground detection, interest-point detection or tracking.