# HiH: A Multi-modal Hierarchy in Hierarchy Network for Unconstrained Gait Recognition

arXiv: https://arxiv.org/abs/2311.11210 · HTML: https://mlsanity.com/paper/2311.11210/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 60 references that are in the map.
- First posted on arXiv 19 Nov 2023.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Bag of Tricks and A Strong Baseline for Deep Person Re-identification](https://mlsanity.com/paper/1903.07071/) (2019)
- [Vision Transformer with Deformable Attention](https://mlsanity.com/paper/2201.00520/) (2022)
- [GaitSet: Regarding Gait as a Set for Cross-View Gait Recognition](https://mlsanity.com/paper/1811.06186/) (2018)
- [Gait Recognition via Effective Global-Local Feature Representation and Local Temporal Aggregation](https://mlsanity.com/paper/2011.01461/) (2020)
- [GaitGraph: Graph Convolutional Network for Skeleton-Based Gait Recognition](https://mlsanity.com/paper/2101.11228/) (2021)
- [Gait Recognition in the Wild with Dense 3D Representations and A Benchmark](https://mlsanity.com/paper/2204.02569/) (2022)
- [OpenGait: Revisiting Gait Recognition Toward Better Practicality](https://mlsanity.com/paper/2211.06597/) (2022)
- [Gait Recognition via Disentangled Representation Learning](https://mlsanity.com/paper/1904.04925/) (2019)
- [Towards a Deeper Understanding of Skeleton-based Gait Recognition](https://mlsanity.com/paper/2204.07855/) (2022)
- [Deep Gait Recognition: A Survey](https://mlsanity.com/paper/2102.09546/) (2021)
- [GaitEdge: Beyond Plain End-to-end Gait Recognition for Better Practicality](https://mlsanity.com/paper/2203.03972/) (2022)
- [GPGait: Generalized Pose-based Gait Recognition](https://mlsanity.com/paper/2303.05234/) (2023)
- [DyGait: Exploiting Dynamic Representations for High-performance Gait Recognition](https://mlsanity.com/paper/2303.14953/) (2023)
- [Parsing is All You Need for Accurate Gait Recognition in the Wild](https://mlsanity.com/paper/2308.16739/) (2023)
- [GaitGCI: Generative Counterfactual Intervention for Gait Recognition](https://mlsanity.com/paper/2306.03428/) (2023)
- [Learning Rich Features for Gait Recognition by Integrating Skeletons and Silhouettes](https://mlsanity.com/paper/2110.13408/) (2021)
- [Gait Recognition in the Wild with Multi-hop Temporal Switch](https://mlsanity.com/paper/2209.00355/) (2022)
- [Gait Recognition Based on Deep Learning: A Survey](https://mlsanity.com/paper/2201.03323/) (2022)
- [Iwin: Human-Object Interaction Detection via Transformer with Irregular Windows](https://mlsanity.com/paper/2203.10537/) (2022)
- [Hierarchical Spatio-Temporal Representation Learning for Gait Recognition](https://mlsanity.com/paper/2307.09856/) (2023)
- [MetaGait: Learning to Learn an Omni Sample Adaptive Representation for Gait Recognition](https://mlsanity.com/paper/2306.03445/) (2023)
- [GaitRef: Gait Recognition with Refined Sequential Skeletons](https://mlsanity.com/paper/2304.07916/) (2023)
- [GaitMM: Multi-Granularity Motion Sequence Learning for Gait Recognition](https://mlsanity.com/paper/2209.08470/) (2022)

## Built on by (most cited first, 4 of 4)

- [GaitSnippet: Gait Recognition Beyond Unordered Sets and Ordered Sequences](https://mlsanity.com/paper/2508.07782/) (2025)
- [Vocabulary-Guided Gait Recognition](https://mlsanity.com/paper/2609.18413/) (2026)
- [Learning Geometric Invariance for Gait Recognition](https://mlsanity.com/paper/2601.05604/) (2026)
- [Learning A Unified Template for Gait Recognition](https://mlsanity.com/paper/2609.18490/) (2026)

## Abstract (arXiv)

Gait recognition has achieved promising advances in controlled settings, yet it significantly struggles in unconstrained environments due to challenges such as view changes, occlusions, and varying walking speeds. Additionally, efforts to fuse multiple modalities often face limited improvements because of cross-modality incompatibility, particularly in outdoor scenarios. To address these issues, we present a multi-modal Hierarchy in Hierarchy network (HiH) that integrates silhouette and pose sequences for robust gait recognition. HiH features a main branch that utilizes Hierarchical Gait Decomposer (HGD) modules for depth-wise and intra-module hierarchical examination of general gait patterns from silhouette data. This approach captures motion hierarchies from overall body dynamics to detailed limb movements, facilitating the representation of gait attributes across multiple spatial resolutions. Complementing this, an auxiliary branch, based on 2D joint sequences, enriches the spatial and temporal aspects of gait analysis. It employs a Deformable Spatial Enhancement (DSE) module for pose-guided spatial attention and a Deformable Temporal Alignment (DTA) module for aligning motion dynamics through learned temporal offsets. Extensive evaluations across diverse indoor and outdoor datasets demonstrate HiH's state-of-the-art performance, affirming a well-balanced trade-off between accuracy and efficiency.