# MMGait: Benchmarking and Unifying Gait Recognition across Heterogeneous Modalities

arXiv: https://arxiv.org/abs/2609.11601 · HTML: https://mlsanity.com/paper/2609.11601/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 37 of its 61 references that are in the map.
- First posted on arXiv 10 Sep 2026.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Bag of Tricks and A Strong Baseline for Deep Person Re-identification](https://mlsanity.com/paper/1903.07071/) (2019)
- [Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks](https://mlsanity.com/paper/1711.10305/) (2017)
- [Domain-Specific Batch Normalization for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1906.03950/) (2019)
- [GaitSet: Regarding Gait as a Set for Cross-View Gait Recognition](https://mlsanity.com/paper/1811.06186/) (2018)
- [Gait Recognition in the Wild with Dense 3D Representations and A Benchmark](https://mlsanity.com/paper/2204.02569/) (2022)
- [OpenGait: Revisiting Gait Recognition Toward Better Practicality](https://mlsanity.com/paper/2211.06597/) (2022)
- [Towards a Deeper Understanding of Skeleton-based Gait Recognition](https://mlsanity.com/paper/2204.07855/) (2022)
- [LidarGait: Benchmarking 3D Gait Recognition with Point Clouds](https://mlsanity.com/paper/2211.10598/) (2022)
- [Deep Gait Recognition: A Survey](https://mlsanity.com/paper/2102.09546/) (2021)
- [Patchwork++: Fast and Robust Ground Segmentation Solving Partial Under-Segmentation Using 3D Point Cloud](https://mlsanity.com/paper/2207.11919/) (2022)
- [Exploring Deep Models for Practical Gait Recognition](https://mlsanity.com/paper/2303.03301/) (2023)
- [BigGait: Learning Gait Representation You Want by Large Vision Models](https://mlsanity.com/paper/2402.19122/) (2024)
- [SkeletonGait: Gait Recognition Using Skeleton Maps](https://mlsanity.com/paper/2311.13444/) (2023)
- [GPGait: Generalized Pose-based Gait Recognition](https://mlsanity.com/paper/2303.05234/) (2023)
- [PaddleSeg: A High-Efficient Development Toolkit for Image Segmentation](https://mlsanity.com/paper/2101.06175/) (2021)
- [Spatial Transformer Network on Skeleton-based Gait Recognition](https://mlsanity.com/paper/2204.03873/) (2022)
- [GaitGL: Learning Discriminative Global-Local Feature Representations for Gait Recognition](https://mlsanity.com/paper/2208.01380/) (2022)
- [Cross-Covariate Gait Recognition: A Benchmark](https://mlsanity.com/paper/2312.14404/) (2023)
- [BiggerGait: Unlocking Gait Recognition with Layer-wise Representations from Large Vision Models](https://mlsanity.com/paper/2505.18132/) (2025)
- [Recurrent Attention Models for Depth-Based Person Identification](https://mlsanity.com/paper/1611.07212/) (2016)
- [All in One Framework for Multimodal Re-identification in the Wild](https://mlsanity.com/paper/2405.04741/) (2024)
- [Gait Recognition in Large-scale Free Environment via Single LiDAR](https://mlsanity.com/paper/2211.12371/) (2022)
- [Causality-inspired Discriminative Feature Learning in Triple Domains for Gait Recognition](https://mlsanity.com/paper/2407.12519/) (2024)
- [Disentangled Diffusion-Based 3D Human Pose Estimation with Hierarchical Spatial and Temporal Denoiser](https://mlsanity.com/paper/2403.04444/) (2024)
- [Exploring More from Multiple Gait Modalities for Human Identification](https://mlsanity.com/paper/2412.11495/) (2024)
- [Camera-LiDAR Cross-modality Gait Recognition](https://mlsanity.com/paper/2407.02038/) (2024)

## Abstract (arXiv)

Gait recognition is commonly studied using RGB videos or their derived silhouettes and poses. Yet human walking produces heterogeneous photometric, geometric, and motion cues that cannot be systematically examined with RGB-centered benchmarks. We present MMGait, a large-scale multi-sensor benchmark that brings visible, infrared, depth, LiDAR, and radar observations into sequence-level correspondence. It provides diverse modalities spanning appearance, contours, geometry, motion, and body structure. Under a shared impostor-augmented protocol, we evaluate single-modal recognition, cross-modal recognition via directed retrieval, and multi-modal recognition using task-specific experts. Across settings, modality rankings vary with probe conditions, cross-modal alignment remains difficult, and fusion often provides complementary gains. This analysis exposes a scalability problem: individual modalities, modality pairs, and fusion configurations are typically handled by separately trained experts. We formulate Omni-Modal Gait Recognition, which unifies single-modal, cross-modal, and multi-modal recognition within a shared identity space. OmniGait++ uses modality-specific front ends followed by a shared identity encoder to preserve modality-dependent cues while learning comparable identity descriptors. An anchor-guided fusion module aggregates modality subsets of varying size without frame-level synchronization. A jointly trained checkpoint covers all three recognition settings and accommodates modality subsets of different compositions and cardinalities. Experiments show OmniGait++ remains competitive with task-specific experts in many shared settings and extends to higher-cardinality fusion unavailable to fixed-pair models. The results establish MMGait as a common testbed for heterogeneous gait sensing and demonstrate the feasibility of unified recognition under varying modality availability.