# Associated Spatio-Temporal Capsule Network for Gait Recognition

arXiv: https://arxiv.org/abs/2101.02458 · HTML: https://mlsanity.com/paper/2101.02458/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 46 references that are in the map.
- First posted on arXiv 7 Jan 2021.

## Builds on (most cited first, 7 of 7)

- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [3D Point Capsule Networks](https://mlsanity.com/paper/1812.10775/) (2018)
- [Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates](https://mlsanity.com/paper/1706.08276/) (2017)
- [Hierarchical Long Short-Term Concurrent Memory for Human Interaction Recognition](https://mlsanity.com/paper/1811.00270/) (2018)
- [Automated Latent Fingerprint Recognition](https://mlsanity.com/paper/1704.01925/) (2017)
- [CapProNet: Deep Feature Learning via Orthogonal Projections onto Capsule Subspaces](https://mlsanity.com/paper/1805.07621/) (2018)

## Built on by (most cited first, 6 of 6)

- [A Comprehensive Survey on Deep Gait Recognition: Algorithms, Datasets and Challenges](https://mlsanity.com/paper/2206.13732/) (2022)
- [Gait Recognition Based on Deep Learning: A Survey](https://mlsanity.com/paper/2201.03323/) (2022)
- [A Methodological and Structural Review of Parkinsons Disease Detection Across Diverse Data Modalities](https://mlsanity.com/paper/2505.00525/) (2025)
- [Progressive Feature Learning for Realistic Cloth-Changing Gait Recognition](https://mlsanity.com/paper/2207.11720/) (2022)
- [Artificial intelligence-enabled detection and assessment of Parkinson's disease using multimodal data: A survey](https://mlsanity.com/paper/2502.10703/) (2025)
- [Watch Where You Move: Region-aware Dynamic Aggregation and Excitation for Gait Recognition](https://mlsanity.com/paper/2510.16541/) (2025)

## Abstract (arXiv)

It is a challenging task to identify a person based on her/his gait patterns. State-of-the-art approaches rely on the analysis of temporal or spatial characteristics of gait, and gait recognition is usually performed on single modality data (such as images, skeleton joint coordinates, or force signals). Evidence has shown that using multi-modality data is more conducive to gait research. Therefore, we here establish an automated learning system, with an associated spatio-temporal capsule network (ASTCapsNet) trained on multi-sensor datasets, to analyze multimodal information for gait recognition. Specifically, we first design a low-level feature extractor and a high-level feature extractor for spatio-temporal feature extraction of gait with a novel recurrent memory unit and a relationship layer. Subsequently, a Bayesian model is employed for the decision-making of class labels. Extensive experiments on several public datasets (normal and abnormal gait) validate the effectiveness of the proposed ASTCapsNet, compared against several state-of-the-art methods.