# Multimodal feature fusion for CNN-based gait recognition: an empirical comparison

arXiv: https://arxiv.org/abs/1806.07753 · HTML: https://mlsanity.com/paper/1806.07753/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2018.

## Built on by (most cited first, 14 of 14)

- [A Comprehensive Survey on Deep Gait Recognition: Algorithms, Datasets and Challenges](https://mlsanity.com/paper/2206.13732/) (2022)
- [GaitGCI: Generative Counterfactual Intervention for Gait Recognition](https://mlsanity.com/paper/2306.03428/) (2023)
- [Gait Recognition in Large-scale Free Environment via Single LiDAR](https://mlsanity.com/paper/2211.12371/) (2022)
- [A Multi-Stage Adaptive Feature Fusion Neural Network for Multimodal Gait Recognition](https://mlsanity.com/paper/2312.14410/) (2023)
- [Cross-Modality Gait Recognition: Bridging LiDAR and Camera Modalities for Human Identification](https://mlsanity.com/paper/2404.04120/) (2024)
- [Multimodal Gait Recognition for Neurodegenerative Diseases](https://mlsanity.com/paper/2101.02469/) (2021)
- [GaitContour: Efficient Gait Recognition based on a Contour-Pose Representation](https://mlsanity.com/paper/2311.16497/) (2023)
- [Watch Where You Move: Region-aware Dynamic Aggregation and Excitation for Gait Recognition](https://mlsanity.com/paper/2510.16541/) (2025)
- [InstaIndoor and Multi-modal Deep Learning for Indoor Scene Recognition](https://mlsanity.com/paper/2112.12409/) (2021)
- [DepthGait: Multi-Scale Cross-Level Feature Fusion of RGB-Derived Depth and Silhouette Sequences for Robust Gait Recognition](https://mlsanity.com/paper/2508.03397/) (2025)
- [Towards Relaxed Multimodal Inputs for Gait-based Parkinson's Disease Assessment](https://mlsanity.com/paper/2510.15748/) (2025)
- [Gait Identification under Surveillance Environment based on Human Skeleton](https://mlsanity.com/paper/2111.11720/) (2021)
- [Spatio-temporal Gait Feature with Global Distance Alignment](https://mlsanity.com/paper/2203.03376/) (2022)
- [Explicit Time-Frequency Dynamics for Skeleton-Based Gait Recognition](https://mlsanity.com/paper/2604.03002/) (2026)

## Abstract (arXiv)

People identification in video based on the way they walk (i.e. gait) is a relevant task in computer vision using a non-invasive approach. Standard and current approaches typically derive gait signatures from sequences of binary energy maps of subjects extracted from images, but this process introduces a large amount of non-stationary noise, thus, conditioning their efficacy. In contrast, in this paper we focus on the raw pixels, or simple functions derived from them, letting advanced learning techniques to extract relevant features. Therefore, we present a comparative study of different Convolutional Neural Network (CNN) architectures by using three different modalities (i.e. gray pixels, optical flow channels and depth maps) on two widely-adopted and challenging datasets: TUM-GAID and CASIA-B. In addition, we perform a comparative study between different early and late fusion methods used to combine the information obtained from each kind of modalities. Our experimental results suggest that (i) the raw pixel values represent a competitive input modality, compared to the traditional state-of-the-art silhouette-based features (e.g. GEI), since equivalent or better results are obtained; (ii) the fusion of the raw pixel information with information from optical flow and depth maps allows to obtain state-of-the-art results on the gait recognition task with an image resolution several times smaller than the previously reported results; and, (iii) the selection and the design of the CNN architecture are critical points that can make a difference between state-of-the-art results or poor ones.