# ZipGait: Bridging Skeleton and Silhouette with Diffusion Model for Advancing Gait Recognition

arXiv: https://arxiv.org/abs/2408.12111 · HTML: https://mlsanity.com/paper/2408.12111/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 25 of its 44 references that are in the map.
- First posted on arXiv 22 Aug 2024.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [DiffusionDet: Diffusion Model for Object Detection](https://mlsanity.com/paper/2211.09788/) (2022)
- [Gait Recognition via Effective Global-Local Feature Representation and Local Temporal Aggregation](https://mlsanity.com/paper/2011.01461/) (2020)
- [GaitGraph: Graph Convolutional Network for Skeleton-Based Gait Recognition](https://mlsanity.com/paper/2101.11228/) (2021)
- [Gait Recognition in the Wild with Dense 3D Representations and A Benchmark](https://mlsanity.com/paper/2204.02569/) (2022)
- [Horizontal Pyramid Matching for Person Re-identification](https://mlsanity.com/paper/1804.05275/) (2018)
- [LidarGait: Benchmarking 3D Gait Recognition with Point Clouds](https://mlsanity.com/paper/2211.10598/) (2022)
- [DyGait: Exploiting Dynamic Representations for High-performance Gait Recognition](https://mlsanity.com/paper/2303.14953/) (2023)
- [GaitGCI: Generative Counterfactual Intervention for Gait Recognition](https://mlsanity.com/paper/2306.03428/) (2023)
- [Learning Rich Features for Gait Recognition by Integrating Skeletons and Silhouettes](https://mlsanity.com/paper/2110.13408/) (2021)
- [Gait Recognition in the Wild with Multi-hop Temporal Switch](https://mlsanity.com/paper/2209.00355/) (2022)
- [Cross-Covariate Gait Recognition: A Benchmark](https://mlsanity.com/paper/2312.14404/) (2023)
- [GaitMixer: Skeleton-based Gait Representation Learning via Wide-spectrum Multi-axial Mixer](https://mlsanity.com/paper/2210.15491/) (2022)
- [DiffuVolume: Diffusion Model for Volume based Stereo Matching](https://mlsanity.com/paper/2308.15989/) (2023)
- [QAGait: Revisit Gait Recognition from a Quality Perspective](https://mlsanity.com/paper/2401.13531/) (2024)
- [GaitContour: Efficient Gait Recognition based on a Contour-Pose Representation](https://mlsanity.com/paper/2311.16497/) (2023)
- [HybridGait: A Benchmark for Spatial-Temporal Cloth-Changing Gait Recognition with Hybrid Explorations](https://mlsanity.com/paper/2401.00271/) (2023)
- [CLASH: Complementary Learning with Neural Architecture Search for Gait Recognition](https://mlsanity.com/paper/2407.03632/) (2024)
- [GaitMA: Pose-guided Multi-modal Feature Fusion for Gait Recognition](https://mlsanity.com/paper/2407.14812/) (2024)

## Abstract (arXiv)

Current gait recognition research predominantly focuses on extracting appearance features effectively, but the performance is severely compromised by the vulnerability of silhouettes under unconstrained scenes. Consequently, numerous studies have explored how to harness information from various models, particularly by sufficiently utilizing the intrinsic information of skeleton sequences. While these model-based methods have achieved significant performance, there is still a huge gap compared to appearance-based methods, which implies the potential value of bridging silhouettes and skeletons. In this work, we make the first attempt to reconstruct dense body shapes from discrete skeleton distributions via the diffusion model, demonstrating a new approach that connects cross-modal features rather than focusing solely on intrinsic features to improve model-based methods. To realize this idea, we propose a novel gait diffusion model named DiffGait, which has been designed with four specific adaptations suitable for gait recognition. Furthermore, to effectively utilize the reconstructed silhouettes and skeletons, we introduce Perception Gait Integration (PGI) to integrate different gait features through a two-stage process. Incorporating those modifications leads to an efficient model-based gait recognition framework called ZipGait. Through extensive experiments on four public benchmarks, ZipGait demonstrates superior performance, outperforming the state-of-the-art methods by a large margin under both cross-domain and intra-domain settings, while achieving significant plug-and-play performance improvements.