# MR-STGN: Multi-Residual Spatio Temporal Graph Network Using Attention Fusion for Patient Action Assessment

arXiv: https://arxiv.org/abs/2312.13509 · HTML: https://mlsanity.com/paper/2312.13509/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 28 references that are in the map.
- First posted on arXiv 21 Dec 2023.

## Builds on (most cited first, 5 of 5)

- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation](https://mlsanity.com/paper/1804.06055/) (2018)
- [A Deep Learning Framework for Assessing Physical Rehabilitation Exercises](https://mlsanity.com/paper/1901.10435/) (2019)
- [Shoulder Physiotherapy Exercise Recognition: Machine Learning the Inertial Signals from a Smartwatch](https://mlsanity.com/paper/1802.01489/) (2018)

## Built on by (most cited first, 5 of 5)

- [An Improved 3D Skeletons UP-Fall Dataset: Enhancing Data Quality for Efficient Impact Fall Detection](https://mlsanity.com/paper/2502.19048/) (2025)
- [Rehabilitation Exercise Quality Assessment through Supervised Contrastive Learning with Hard and Soft Negatives](https://mlsanity.com/paper/2403.02772/) (2024)
- [Leveraging Auto-Distillation and Generative Self-Supervised Learning in Residual Graph Transformers for Enhanced Recommender Systems](https://mlsanity.com/paper/2504.10500/) (2025)
- [AI-Based Stroke Rehabilitation Domiciliary Assessment System with ST_GCN Attention](https://mlsanity.com/paper/2510.00049/) (2025)
- [A Point Cloud Transformer for Remote Monitoring and Automated Assessment of Physical Rehabilitation Exercises](https://mlsanity.com/paper/2606.30309/) (2026)

## Abstract (arXiv)

Accurate assessment of patient actions plays a crucial role in healthcare as it contributes significantly to disease progression monitoring and treatment effectiveness. However, traditional approaches to assess patient actions often rely on manual observation and scoring, which are subjective and time-consuming. In this paper, we propose an automated approach for patient action assessment using a Multi-Residual Spatio Temporal Graph Network (MR-STGN) that incorporates both angular and positional 3D skeletons. The MR-STGN is specifically designed to capture the spatio-temporal dynamics of patient actions. It achieves this by integrating information from multiple residual layers, with each layer extracting features at distinct levels of abstraction. Furthermore, we integrate an attention fusion mechanism into the network, which facilitates the adaptive weighting of various features. This empowers the model to concentrate on the most pertinent aspects of the patient's movements, offering precise instructions regarding specific body parts or movements that require attention. Ablation studies are conducted to analyze the impact of individual components within the proposed model. We evaluate our model on the UI-PRMD dataset demonstrating its performance in accurately predicting real-time patient action scores, surpassing state-of-the-art methods.