# Learning with Muscles: Benefits for Data-Efficiency and Robustness in Anthropomorphic Tasks

arXiv: https://arxiv.org/abs/2207.03952 · HTML: https://mlsanity.com/paper/2207.03952/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 61 references that are in the map.
- First posted on arXiv 8 Jul 2022.

## Builds on (most cited first, 14 of 14)

- [Maximum a Posteriori Policy Optimisation](https://mlsanity.com/paper/1806.06920/) (2018)
- [MyoSuite -- A contact-rich simulation suite for musculoskeletal motor control](https://mlsanity.com/paper/2205.13600/) (2022)
- [Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?](https://mlsanity.com/paper/1611.01055/) (2016)
- [DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systems](https://mlsanity.com/paper/2206.00484/) (2022)
- [Learning to Locomote: Understanding How Environment Design Matters for Deep Reinforcement Learning](https://mlsanity.com/paper/2010.04304/) (2020)
- [Learning to Run challenge solutions: Adapting reinforcement learning methods for neuromusculoskeletal environments](https://mlsanity.com/paper/1804.00361/) (2018)
- [Tonic: A Deep Reinforcement Learning Library for Fast Prototyping and Benchmarking](https://mlsanity.com/paper/2011.07537/) (2020)
- [Bayesian Optimization Using Domain Knowledge on the ATRIAS Biped](https://mlsanity.com/paper/1709.06047/) (2017)
- [Variable impedance control and learning -- A review](https://mlsanity.com/paper/2010.06246/) (2020)
- [Artificial Intelligence for Prosthetics - challenge solutions](https://mlsanity.com/paper/1902.02441/) (2019)
- [Reinforcement Learning Control of a Biomechanical Model of the Upper Extremity](https://mlsanity.com/paper/2011.07105/) (2020)
- [OstrichRL: A Musculoskeletal Ostrich Simulation to Study Bio-mechanical Locomotion](https://mlsanity.com/paper/2112.06061/) (2021)
- [Reinforcement Learning of Musculoskeletal Control from Functional Simulations](https://mlsanity.com/paper/2007.06669/) (2020)
- [Learning to Control Highly Accelerated Ballistic Movements on Muscular Robots](https://mlsanity.com/paper/1904.03665/) (2019)

## Built on by (most cited first, 6 of 6)

- [Natural and Robust Walking using Reinforcement Learning without Demonstrations in High-Dimensional Musculoskeletal Models](https://mlsanity.com/paper/2309.02976/) (2023)
- [Investigating the Impact of Action Representations in Policy Gradient Algorithms](https://mlsanity.com/paper/2309.06921/) (2023)
- [MIMo: A Multi-Modal Infant Model for Studying Cognitive Development](https://mlsanity.com/paper/2312.04318/) (2023)
- [Arnold: A multi-task, multi-embodiment muscle transformer policy](https://mlsanity.com/paper/2508.18066/) (2025)
- [Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networks](https://mlsanity.com/paper/2604.09487/) (2026)
- [Model-Based Reinforcement Learning Exploits Passive Body Dynamics for High-Performance Biped Robot Locomotion](https://mlsanity.com/paper/2604.14565/) (2026)

## Abstract (arXiv)

Humans are able to outperform robots in terms of robustness, versatility, and learning of new tasks in a wide variety of movements. We hypothesize that highly nonlinear muscle dynamics play a large role in providing inherent stability, which is favorable to learning. While recent advances have been made in applying modern learning techniques to muscle-actuated systems both in simulation as well as in robotics, so far, no detailed analysis has been performed to show the benefits of muscles when learning from scratch. Our study closes this gap and showcases the potential of muscle actuators for core robotics challenges in terms of data-efficiency, hyperparameter sensitivity, and robustness.