# Deep learning-based approaches for human motion decoding in smart walkers for rehabilitation

arXiv: https://arxiv.org/abs/2301.05575 · HTML: https://mlsanity.com/paper/2301.05575/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 57 references that are in the map.
- First posted on arXiv 13 Jan 2023.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Video Action Transformer Network](https://mlsanity.com/paper/1812.02707/) (2018)
- [RED: Reinforced Encoder-Decoder Networks for Action Anticipation](https://mlsanity.com/paper/1707.04818/) (2017)
- [Encouraging LSTMs to Anticipate Actions Very Early](https://mlsanity.com/paper/1703.07023/) (2017)
- [Online Human Action Detection using Joint Classification-Regression Recurrent Neural Networks](https://mlsanity.com/paper/1604.05633/) (2016)
- [Action Anticipation with RBF Kernelized Feature Mapping RNN](https://mlsanity.com/paper/1911.07806/) (2019)
- [Lightweight Network Architecture for Real-Time Action Recognition](https://mlsanity.com/paper/1905.08711/) (2019)
- [Action Anticipation for Collaborative Environments: The Impact of Contextual Information and Uncertainty-Based Prediction](https://mlsanity.com/paper/1910.00714/) (2019)
- [Rethinking Online Action Detection in Untrimmed Videos: A Novel Online Evaluation Protocol](https://mlsanity.com/paper/2003.12041/) (2020)
- [Real-Time Human Pose Estimation on a Smart Walker using Convolutional Neural Networks](https://mlsanity.com/paper/2106.14739/) (2021)

## Built on by (most cited first, 1 of 1)

- [Adversarially-Refined VQ-GAN with Dense Motion Tokenization for Spatio-Temporal Heatmaps](https://mlsanity.com/paper/2509.19252/) (2025)

## Abstract (arXiv)

Gait disabilities are among the most frequent worldwide. Their treatment relies on rehabilitation therapies, in which smart walkers are being introduced to empower the user's recovery and autonomy, while reducing the clinicians effort. For that, these should be able to decode human motion and needs, as early as possible. Current walkers decode motion intention using information of wearable or embedded sensors, namely inertial units, force and hall sensors, and lasers, whose main limitations imply an expensive solution or hinder the perception of human movement. Smart walkers commonly lack a seamless human-robot interaction, which intuitively understands human motions. A contactless approach is proposed in this work, addressing human motion decoding as an early action recognition/detection problematic, using RGB-D cameras. We studied different deep learning-based algorithms, organised in three different approaches, to process lower body RGB-D video sequences, recorded from an embedded camera of a smart walker, and classify them into 4 classes (stop, walk, turn right/left). A custom dataset involving 15 healthy participants walking with the device was acquired and prepared, resulting in 28800 balanced RGB-D frames, to train and evaluate the deep networks. The best results were attained by a convolutional neural network with a channel attention mechanism, reaching accuracy values of 99.61% and above 93%, for offline early detection/recognition and trial simulations, respectively. Following the hypothesis that human lower body features encode prominent information, fostering a more robust prediction towards real-time applications, the algorithm focus was also evaluated using Dice metric, leading to values slightly higher than 30%. Promising results were attained for early action detection as a human motion decoding strategy, with enhancements in the focus of the proposed architectures.