# GaitMA: Pose-guided Multi-modal Feature Fusion for Gait Recognition

arXiv: https://arxiv.org/abs/2407.14812 · HTML: https://mlsanity.com/paper/2407.14812/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 20 Jul 2024.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Revisiting Skeleton-based Action Recognition](https://mlsanity.com/paper/2104.13586/) (2021)
- [Learning with a Wasserstein Loss](https://mlsanity.com/paper/1506.05439/) (2015)
- [GaitSet: Regarding Gait as a Set for Cross-View Gait Recognition](https://mlsanity.com/paper/1811.06186/) (2018)
- [Gait Recognition via Effective Global-Local Feature Representation and Local Temporal Aggregation](https://mlsanity.com/paper/2011.01461/) (2020)
- [GaitGraph: Graph Convolutional Network for Skeleton-Based Gait Recognition](https://mlsanity.com/paper/2101.11228/) (2021)
- [Gait Recognition in the Wild with Dense 3D Representations and A Benchmark](https://mlsanity.com/paper/2204.02569/) (2022)
- [OpenGait: Revisiting Gait Recognition Toward Better Practicality](https://mlsanity.com/paper/2211.06597/) (2022)
- [Horizontal Pyramid Matching for Person Re-identification](https://mlsanity.com/paper/1804.05275/) (2018)
- [Towards a Deeper Understanding of Skeleton-based Gait Recognition](https://mlsanity.com/paper/2204.07855/) (2022)
- [Deep Gait Recognition: A Survey](https://mlsanity.com/paper/2102.09546/) (2021)
- [GPGait: Generalized Pose-based Gait Recognition](https://mlsanity.com/paper/2303.05234/) (2023)
- [Spatial Transformer Network on Skeleton-based Gait Recognition](https://mlsanity.com/paper/2204.03873/) (2022)
- [Learning Rich Features for Gait Recognition by Integrating Skeletons and Silhouettes](https://mlsanity.com/paper/2110.13408/) (2021)

## Built on by (most cited first, 2 of 2)

- [PGcGAN: Pathological Gait-Conditioned GAN for Human Gait Synthesis](https://mlsanity.com/paper/2603.14409/) (2026)
- [ZipGait: Bridging Skeleton and Silhouette with Diffusion Model for Advancing Gait Recognition](https://mlsanity.com/paper/2408.12111/) (2024)

## Abstract (arXiv)

Gait recognition is a biometric technology that recognizes the identity of humans through their walking patterns. Existing appearance-based methods utilize CNN or Transformer to extract spatial and temporal features from silhouettes, while model-based methods employ GCN to focus on the special topological structure of skeleton points. However, the quality of silhouettes is limited by complex occlusions, and skeletons lack dense semantic features of the human body. To tackle these problems, we propose a novel gait recognition framework, dubbed Gait Multi-model Aggregation Network (GaitMA), which effectively combines two modalities to obtain a more robust and comprehensive gait representation for recognition. First, skeletons are represented by joint/limb-based heatmaps, and features from silhouettes and skeletons are respectively extracted using two CNN-based feature extractors. Second, a co-attention alignment module is proposed to align the features by element-wise attention. Finally, we propose a mutual learning module, which achieves feature fusion through cross-attention, Wasserstein loss is further introduced to ensure the effective fusion of two modalities. Extensive experimental results demonstrate the superiority of our model on Gait3D, OU-MVLP, and CASIA-B.