# Towards AI-controlled FES-restoration of movements: Learning cycling stimulation pattern with reinforcement learning

arXiv: https://arxiv.org/abs/2303.09986 · HTML: https://mlsanity.com/paper/2303.09986/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 31 references that are in the map.
- First posted on arXiv 17 Mar 2023.

## Builds on (most cited first, 9 of 9)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Gaze-based, Context-aware Robotic System for Assisted Reaching and Grasping](https://mlsanity.com/paper/1809.08095/) (2018)
- [Reinforcement Learning Control of a Biomechanical Model of the Upper Extremity](https://mlsanity.com/paper/2011.07105/) (2020)
- [Neuromechanics-based Deep Reinforcement Learning of Neurostimulation Control in FES cycling](https://mlsanity.com/paper/2103.03057/) (2021)
- [Neuromuscular Reinforcement Learning to Actuate Human Limbs through FES](https://mlsanity.com/paper/2209.07849/) (2022)
- [Towards AI-controlled FES-restoration of arm movements: neuromechanics-based reinforcement learning for 3-D reaching](https://mlsanity.com/paper/2301.04004/) (2023)
- [Towards AI-controlled FES-restoration of arm movements: Controlling for progressive muscular fatigue with Gaussian state-space models](https://mlsanity.com/paper/2301.04005/) (2023)

## Abstract (arXiv)

Functional electrical stimulation (FES) has been increasingly integrated with other rehabilitation devices, including robots. FES cycling is one of the common FES applications in rehabilitation, which is performed by stimulating leg muscles in a certain pattern. The appropriate pattern varies across individuals and requires manual tuning which can be time-consuming and challenging for the individual user. Here, we present an AI-based method for finding the patterns, which requires no extra hardware or sensors. Our method has two phases, starting with finding model-based patterns using reinforcement learning and detailed musculoskeletal models. The models, built using open-source software, can be customised through our automated script and can be therefore used by non-technical individuals without extra cost. Next, our method fine-tunes the pattern using real cycling data. We test our both in simulation and experimentally on a stationary tricycle. In the simulation test, our method can robustly deliver model-based patterns for different cycling configurations. The experimental evaluation shows that our method can find a model-based pattern that induces higher cycling speed than an EMG-based pattern. By using just 100 seconds of cycling data, our method can deliver a fine-tuned pattern that gives better cycling performance. Beyond FES cycling, this work is a showcase, displaying the feasibility and potential of human-in-the-loop AI in real-world rehabilitation.