# A Point Cloud Transformer for Remote Monitoring and Automated Assessment of Physical Rehabilitation Exercises

arXiv: https://arxiv.org/abs/2606.30309 · HTML: https://mlsanity.com/paper/2606.30309/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 74 references that are in the map.
- First posted on arXiv 29 Jun 2026.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting](https://mlsanity.com/paper/1709.04875/) (2017)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds](https://mlsanity.com/paper/1911.11236/) (2019)
- [Graph U-Nets](https://mlsanity.com/paper/1905.05178/) (2019)
- [Axial Attention in Multidimensional Transformers](https://mlsanity.com/paper/1912.12180/) (2019)
- [Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation](https://mlsanity.com/paper/2003.07853/) (2020)
- [PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling](https://mlsanity.com/paper/2003.00492/) (2020)
- [Structural-RNN: Deep Learning on Spatio-Temporal Graphs](https://mlsanity.com/paper/1511.05298/) (2015)
- [Walk in the Cloud: Learning Curves for Point Clouds Shape Analysis](https://mlsanity.com/paper/2105.01288/) (2021)
- [Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition](https://mlsanity.com/paper/1904.01189/) (2019)
- [Skeleton-Based Action Recognition with Multi-Stream Adaptive Graph Convolutional Networks](https://mlsanity.com/paper/1912.06971/) (2019)
- [View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognition](https://mlsanity.com/paper/1804.07453/) (2018)
- [An Alternative Probabilistic Interpretation of the Huber Loss](https://mlsanity.com/paper/1911.02088/) (2019)
- [When Human Pose Estimation Meets Robustness: Adversarial Algorithms and Benchmarks](https://mlsanity.com/paper/2105.06152/) (2021)
- [Richly Activated Graph Convolutional Network for Robust Skeleton-based Action Recognition](https://mlsanity.com/paper/2008.03791/) (2020)
- [A Deep Learning Framework for Assessing Physical Rehabilitation Exercises](https://mlsanity.com/paper/1901.10435/) (2019)
- [Space-Time Representation of People Based on 3D Skeletal Data: A Review](https://mlsanity.com/paper/1601.01006/) (2016)
- [Transition Forests: Learning Discriminative Temporal Transitions for Action Recognition and Detection](https://mlsanity.com/paper/1607.02737/) (2016)
- [D-STGCNT: A Dense Spatio-Temporal Graph Conv-GRU Network based on transformer for assessment of patient physical rehabilitation](https://mlsanity.com/paper/2401.06150/) (2023)
- [Shoulder Physiotherapy Exercise Recognition: Machine Learning the Inertial Signals from a Smartwatch](https://mlsanity.com/paper/1802.01489/) (2018)
- [MR-STGN: Multi-Residual Spatio Temporal Graph Network Using Attention Fusion for Patient Action Assessment](https://mlsanity.com/paper/2312.13509/) (2023)
- [Less is More: Facial Landmarks can Recognize a Spontaneous Smile](https://mlsanity.com/paper/2210.04240/) (2022)

## Abstract (arXiv)

Rehabilitation exercises are essential in restoring lost physical functions of patients suffering from various diseases (e.g., Parkinson's, back pain). Carrying out these rehabilitation exercises, often prescribed by health experts, is costly, unavailable, and requires expert supervision. The availability of RGBD images and movement/position data of joints along with expert annotation of exercise data has prompted the use of automatic assessment of the quality of rehabilitation exercises, which is cost-effective and can be carried out at home. However, existing approaches do not extract relevant features, lack practical application, require expensive pre-processing, or overlook crucial features. This study proposes a transformer-based framework for point clouds to extract features and assess rehabilitation exercises by analyzing joint positions collected through RGBD data. We adapt and utilize a curve-based point-cloud feature aggregation technique to augment point-cloud information that aids model output. The transformer architecture also uses axial self-attention, recognizing important joints and their roles to assist users in performing the exercise better. The guided system outperforms existing approaches and is also practically relevant due to its small size, fast inference, and generalization on specific joints in similar exercises. We conduct our experiments on three crucial baseline datasets for rehabilitation exercises: Kimore, UI-PRMD, and IRDS.