# Multimodal Gait Recognition for Neurodegenerative Diseases

arXiv: https://arxiv.org/abs/2101.02469 · HTML: https://mlsanity.com/paper/2101.02469/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 50 references that are in the map.
- First posted on arXiv 7 Jan 2021.

## Builds on (most cited first, 5 of 5)

- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [Multimodal feature fusion for CNN-based gait recognition: an empirical comparison](https://mlsanity.com/paper/1806.07753/) (2018)
- [Robust Gait Recognition by Integrating Inertial and RGBD Sensors](https://mlsanity.com/paper/1610.09816/) (2016)

## Built on by (most cited first, 8 of 8)

- [Cross-Modality Gait Recognition: Bridging LiDAR and Camera Modalities for Human Identification](https://mlsanity.com/paper/2404.04120/) (2024)
- [A Methodological and Structural Review of Parkinsons Disease Detection Across Diverse Data Modalities](https://mlsanity.com/paper/2505.00525/) (2025)
- [Non-contact Multimodal Indoor Human Monitoring Systems: A Survey](https://mlsanity.com/paper/2312.07601/) (2023)
- [Artificial intelligence-enabled detection and assessment of Parkinson's disease using multimodal data: A survey](https://mlsanity.com/paper/2502.10703/) (2025)
- [Towards Relaxed Multimodal Inputs for Gait-based Parkinson's Disease Assessment](https://mlsanity.com/paper/2510.15748/) (2025)
- [VersatileGait: A Large-Scale Synthetic Gait Dataset Towards in-the-Wild Simulation](https://mlsanity.com/paper/2105.14421/) (2021)
- [Patient-Specific Game-Based Transfer Method for Parkinson's Disease Severity Prediction](https://mlsanity.com/paper/2208.04315/) (2022)
- [MOSAIC: Modality-Specific Adaptation for Incremental Continual Learning in Parkinson's Disease Gait Assessment](https://mlsanity.com/paper/2606.13258/) (2026)

## Abstract (arXiv)

In recent years, single modality based gait recognition has been extensively explored in the analysis of medical images or other sensory data, and it is recognised that each of the established approaches has different strengths and weaknesses. As an important motor symptom, gait disturbance is usually used for diagnosis and evaluation of diseases; moreover, the use of multi-modality analysis of the patient's walking pattern compensates for the one-sidedness of single modality gait recognition methods that only learn gait changes in a single measurement dimension. The fusion of multiple measurement resources has demonstrated promising performance in the identification of gait patterns associated with individual diseases. In this paper, as a useful tool, we propose a novel hybrid model to learn the gait differences between three neurodegenerative diseases, between patients with different severity levels of Parkinson's disease and between healthy individuals and patients, by fusing and aggregating data from multiple sensors. A spatial feature extractor (SFE) is applied to generating representative features of images or signals. In order to capture temporal information from the two modality data, a new correlative memory neural network (CorrMNN) architecture is designed for extracting temporal features. Afterwards, we embed a multi-switch discriminator to associate the observations with individual state estimations. Compared with several state-of-the-art techniques, our proposed framework shows more accurate classification results.