# SAR: Generalization of Physiological Agility and Dexterity via Synergistic Action Representation

arXiv: https://arxiv.org/abs/2307.03716 · HTML: https://mlsanity.com/paper/2307.03716/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 74 references that are in the map.
- First posted on arXiv 7 Jul 2023.

## Builds on (most cited first, 10 of 10)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research](https://mlsanity.com/paper/1802.09464/) (2018)
- [A System for General In-Hand Object Re-Orientation](https://mlsanity.com/paper/2111.03043/) (2021)
- [ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging](https://mlsanity.com/paper/1904.06830/) (2019)
- [MyoSuite -- A contact-rich simulation suite for musculoskeletal motor control](https://mlsanity.com/paper/2205.13600/) (2022)
- [Smooth Exploration for Robotic Reinforcement Learning](https://mlsanity.com/paper/2005.05719/) (2020)
- [Dynamic Movement Primitives in Robotics: A Tutorial Survey](https://mlsanity.com/paper/2102.03861/) (2021)
- [DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systems](https://mlsanity.com/paper/2206.00484/) (2022)

## Built on by (most cited first, 17 of 17)

- [Self Model for Embodied Intelligence: Modeling Full-Body Human Musculoskeletal System and Locomotion Control with Hierarchical Low-Dimensional Representation](https://mlsanity.com/paper/2312.05473/) (2023)
- [DynSyn: Dynamical Synergistic Representation for Efficient Learning and Control in Overactuated Embodied Systems](https://mlsanity.com/paper/2407.11472/) (2024)
- [KINESIS: Motion Imitation for Human Musculoskeletal Locomotion](https://mlsanity.com/paper/2503.14637/) (2025)
- [Natural and Robust Walking using Reinforcement Learning without Demonstrations in High-Dimensional Musculoskeletal Models](https://mlsanity.com/paper/2309.02976/) (2023)
- [Scalable Exploration for High-Dimensional Continuous Control via Value-Guided Flow](https://mlsanity.com/paper/2601.19707/) (2026)
- [emg2tendon: From sEMG Signals to Tendon Control in Musculoskeletal Hands](https://mlsanity.com/paper/2508.08269/) (2025)
- [Arnold: A multi-task, multi-embodiment muscle transformer policy](https://mlsanity.com/paper/2508.18066/) (2025)
- [Compositional meta-learning through probabilistic task inference](https://mlsanity.com/paper/2510.01858/) (2025)
- [Exciting Action: Investigating Efficient Exploration for Learning Musculoskeletal Humanoid Locomotion](https://mlsanity.com/paper/2407.11658/) (2024)
- [Bipedal Balance Control with Whole-body Musculoskeletal Standing and Falling Simulations](https://mlsanity.com/paper/2506.09383/) (2025)
- [Musculoskeletal simulation of limb movement biomechanics in Drosophila melanogaster](https://mlsanity.com/paper/2509.06426/) (2025)
- [Log2Motion: Biomechanical Motion Synthesis from Touch Logs](https://mlsanity.com/paper/2601.21043/) (2026)
- [Muscle Synergy Priors Enhance Biomechanical Fidelity in Predictive Musculoskeletal Locomotion Simulation](https://mlsanity.com/paper/2603.10474/) (2026)
- [MUSIC: Learning Muscle-Driven Dexterous Hand Control](https://mlsanity.com/paper/2604.23886/) (2026)
- [Replication of Impedance Identification Experiments on a Reinforcement-Learning-Controlled Digital Twin of Human Elbows](https://mlsanity.com/paper/2402.02904/) (2024)
- [Embodied Learning of Reward for Musculoskeletal Control with Vision Language Models](https://mlsanity.com/paper/2512.23077/) (2025)
- [Lambda-Hold Control: Human-Like Movement Emerges from a Minimal Task Reward in Predictive Musculoskeletal Simulation](https://mlsanity.com/paper/2608.17030/) (2026)

## Abstract (arXiv)

Learning effective continuous control policies in high-dimensional systems, including musculoskeletal agents, remains a significant challenge. Over the course of biological evolution, organisms have developed robust mechanisms for overcoming this complexity to learn highly sophisticated strategies for motor control. What accounts for this robust behavioral flexibility? Modular control via muscle synergies, i.e. coordinated muscle co-contractions, is considered to be one putative mechanism that enables organisms to learn muscle control in a simplified and generalizable action space. Drawing inspiration from this evolved motor control strategy, we use physiologically accurate human hand and leg models as a testbed for determining the extent to which a Synergistic Action Representation (SAR) acquired from simpler tasks facilitates learning more complex tasks. We find in both cases that SAR-exploiting policies significantly outperform end-to-end reinforcement learning. Policies trained with SAR were able to achieve robust locomotion on a wide set of terrains with high sample efficiency, while baseline approaches failed to learn meaningful behaviors. Additionally, policies trained with SAR on a multiobject manipulation task significantly outperformed (>70% success) baseline approaches (<20% success). Both of these SAR-exploiting policies were also found to generalize zero-shot to out-of-domain environmental conditions, while policies that did not adopt SAR failed to generalize. Finally, we establish the generality of SAR on broader high-dimensional control problems using a robotic manipulation task set and a full-body humanoid locomotion task. To the best of our knowledge, this investigation is the first of its kind to present an end-to-end pipeline for discovering synergies and using this representation to learn high-dimensional continuous control across a wide diversity of tasks.