# Reinforcement Learning Control of a Biomechanical Model of the Upper Extremity

arXiv: https://arxiv.org/abs/2011.07105 · HTML: https://mlsanity.com/paper/2011.07105/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Nov 2020.

## Built on by (most cited first, 11 of 11)

- [DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systems](https://mlsanity.com/paper/2206.00484/) (2022)
- [Learning with Muscles: Benefits for Data-Efficiency and Robustness in Anthropomorphic Tasks](https://mlsanity.com/paper/2207.03952/) (2022)
- [Increasing Interaction Fidelity: Training Routines for Biomechanical Models in HCI](https://mlsanity.com/paper/2508.16581/) (2025)
- [Towards AI-controlled FES-restoration of arm movements: neuromechanics-based reinforcement learning for 3-D reaching](https://mlsanity.com/paper/2301.04004/) (2023)
- [Generating Realistic Arm Movements in Reinforcement Learning: A Quantitative Comparison of Reward Terms and Task Requirements](https://mlsanity.com/paper/2402.13949/) (2024)
- [Log2Motion: Biomechanical Motion Synthesis from Touch Logs](https://mlsanity.com/paper/2601.21043/) (2026)
- [What deep reinforcement learning tells us about human motor learning and vice-versa](https://mlsanity.com/paper/2208.10892/) (2022)
- [Towards AI-controlled FES-restoration of movements: Learning cycling stimulation pattern with reinforcement learning](https://mlsanity.com/paper/2303.09986/) (2023)
- [Replication of Impedance Identification Experiments on a Reinforcement-Learning-Controlled Digital Twin of Human Elbows](https://mlsanity.com/paper/2402.02904/) (2024)
- [MyoInteract: A Framework for Fast Prototyping of Biomechanical HCI Tasks using Reinforcement Learning](https://mlsanity.com/paper/2602.15245/) (2026)
- [Designing Fatigue-Aware VR Interfaces via Biomechanical Models](https://mlsanity.com/paper/2603.26031/) (2026)

## Abstract (arXiv)

Among the infinite number of possible movements that can be produced, humans are commonly assumed to choose those that optimize criteria such as minimizing movement time, subject to certain movement constraints like signal-dependent and constant motor noise. While so far these assumptions have only been evaluated for simplified point-mass or planar models, we address the question of whether they can predict reaching movements in a full skeletal model of the human upper extremity. We learn a control policy using a motor babbling approach as implemented in reinforcement learning, using aimed movements of the tip of the right index finger towards randomly placed 3D targets of varying size. We use a state-of-the-art biomechanical model, which includes seven actuated degrees of freedom. To deal with the curse of dimensionality, we use a simplified second-order muscle model, acting at each degree of freedom instead of individual muscles. The results confirm that the assumptions of signal-dependent and constant motor noise, together with the objective of movement time minimization, are sufficient for a state-of-the-art skeletal model of the human upper extremity to reproduce complex phenomena of human movement, in particular Fitts' Law and the 2/3 Power Law. This result supports the notion that control of the complex human biomechanical system can plausibly be determined by a set of simple assumptions and can easily be learned.