# Towards AI-controlled FES-restoration of arm movements: neuromechanics-based reinforcement learning for 3-D reaching

arXiv: https://arxiv.org/abs/2301.04004 · HTML: https://mlsanity.com/paper/2301.04004/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 22 references that are in the map.
- First posted on arXiv 10 Jan 2023.

## Builds on (most cited first, 6 of 6)

- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [Gaze-based, Context-aware Robotic System for Assisted Reaching and Grasping](https://mlsanity.com/paper/1809.08095/) (2018)
- [Reinforcement Learning Control of a Biomechanical Model of the Upper Extremity](https://mlsanity.com/paper/2011.07105/) (2020)
- [Neuromechanics-based Deep Reinforcement Learning of Neurostimulation Control in FES cycling](https://mlsanity.com/paper/2103.03057/) (2021)
- [Neuromuscular Reinforcement Learning to Actuate Human Limbs through FES](https://mlsanity.com/paper/2209.07849/) (2022)

## Built on by (most cited first, 1 of 1)

- [Towards AI-controlled FES-restoration of movements: Learning cycling stimulation pattern with reinforcement learning](https://mlsanity.com/paper/2303.09986/) (2023)

## Abstract (arXiv)

Reaching disabilities affect the quality of life. Functional Electrical Stimulation (FES) can restore lost motor functions. Yet, there remain challenges in controlling FES to induce desired movements. Neuromechanical models are valuable tools for developing FES control methods. However, focusing on the upper extremity areas, several existing models are either overly simplified or too computationally demanding for control purposes. Besides the model-related issues, finding a general method for governing the control rules for different tasks and subjects remains an engineering challenge. Here, we present our approach toward FES-based restoration of arm movements to address those fundamental issues in controlling FES. Firstly, we present our surface-FES-oriented neuromechanical models of human arms built using well-accepted, open-source software. The models are designed to capture significant dynamics in FES controls with minimal computational cost. Our models are customisable and can be used for testing different control methods. Secondly, we present the application of reinforcement learning (RL) as a general method for governing the control rules. In combination, our customisable models and RL-based control method open the possibility of delivering customised FES controls for different subjects and settings with minimal engineering intervention. We demonstrate our approach in planar and 3D settings.