# Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networks

arXiv: https://arxiv.org/abs/2604.09487 · HTML: https://mlsanity.com/paper/2604.09487/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 45 references that are in the map.
- First posted on arXiv 10 Apr 2026.

## Builds on (most cited first, 30 of 31)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Self-Supervised Exploration via Disagreement](https://mlsanity.com/paper/1906.04161/) (2019)
- [Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning](https://mlsanity.com/paper/1907.04490/) (2019)
- [Real-World Humanoid Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2303.03381/) (2023)
- [Model-Based Active Exploration](https://mlsanity.com/paper/1810.12162/) (2018)
- [Robot Learning from Randomized Simulations: A Review](https://mlsanity.com/paper/2111.00956/) (2021)
- [skrl: Modular and Flexible Library for Reinforcement Learning](https://mlsanity.com/paper/2202.03825/) (2022)
- [DribbleBot: Dynamic Legged Manipulation in the Wild](https://mlsanity.com/paper/2304.01159/) (2023)
- [Learning to Play Table Tennis From Scratch using Muscular Robots](https://mlsanity.com/paper/2006.05935/) (2020)
- [HITTER: A HumanoId Table TEnnis Robot via Hierarchical Planning and Learning](https://mlsanity.com/paper/2508.21043/) (2025)
- [Domain Randomization via Entropy Maximization](https://mlsanity.com/paper/2311.01885/) (2023)
- [Learning Sim-to-Real Humanoid Locomotion in 15 Minutes](https://mlsanity.com/paper/2512.01996/) (2025)
- [Bridging the Sim-to-Real Gap for Athletic Loco-Manipulation](https://mlsanity.com/paper/2502.10894/) (2025)
- [Active Exploration for Robotic Manipulation](https://mlsanity.com/paper/2210.12806/) (2022)
- [Learning with Muscles: Benefits for Data-Efficiency and Robustness in Anthropomorphic Tasks](https://mlsanity.com/paper/2207.03952/) (2022)
- [Reliable Real Time Ball Tracking for Robot Table Tennis](https://mlsanity.com/paper/1908.07332/) (2019)
- [Safe & Accurate at Speed with Tendons: A Robot Arm for Exploring Dynamic Motion](https://mlsanity.com/paper/2307.02654/) (2023)
- [Learning to Control Highly Accelerated Ballistic Movements on Muscular Robots](https://mlsanity.com/paper/1904.03665/) (2019)
- [Reinforcement Learning Control for Autonomous Hydraulic Material Handling Machines with Underactuated Tools](https://mlsanity.com/paper/2410.05093/) (2024)
- [Learning Agile Striker Skills for Humanoid Soccer Robots from Noisy Sensory Input](https://mlsanity.com/paper/2512.06571/) (2025)
- [AIMY: An Open-source Table Tennis Ball Launcher for Versatile and High-fidelity Trajectory Generation](https://mlsanity.com/paper/2210.06048/) (2022)
- [Hindsight States: Blending Sim and Real Task Elements for Efficient Reinforcement Learning](https://mlsanity.com/paper/2303.02234/) (2023)
- [Component Modularized Design of Musculoskeletal Humanoid Platform Musashi to Investigate Learning Control Systems](https://mlsanity.com/paper/2410.22000/) (2024)
- [Tendon Force Modeling for Sim2Real Transfer of Reinforcement Learning Policies for Tendon-Driven Robots](https://mlsanity.com/paper/2603.04351/) (2026)
- [An Efficient Learning Control Framework With Sim-to-Real for String-Type Artificial Muscle-Driven Robotic Systems](https://mlsanity.com/paper/2405.10576/) (2024)

## Abstract (arXiv)

Tendon drives paired with soft muscle actuation enable faster and safer robots while potentially accelerating skill acquisition. Still, these systems are rarely used in practice due to inherent nonlinearities, friction, and hysteresis, which complicate modeling and control. So far, these challenges have hindered policy transfer from simulation to real systems. To bridge this gap, we propose a sim-to-real pipeline that learns a neural network model of this complex actuation and leverages established rigid body simulation for the arm dynamics and interactions with the environment. Our method, called Generalized Actuator Network (GenAN), enables actuation model identification across a wide range of robots by learning directly from joint position trajectories rather than requiring torque sensors. Using GenAN on PAMY2, a tendon-driven robot powered by pneumatic artificial muscles, we successfully deploy dynamic but precise goal-reaching, ball-in-a-cup, and table tennis policies, trained entirely in simulation. To the best of our knowledge, this result constitutes the first successful sim-to-real transfer for a four-degrees-of-freedom muscle-actuated robot arm.