# OstrichRL: A Musculoskeletal Ostrich Simulation to Study Bio-mechanical Locomotion

arXiv: https://arxiv.org/abs/2112.06061 · HTML: https://mlsanity.com/paper/2112.06061/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 63 references that are in the map.
- First posted on arXiv 11 Dec 2021.

## Builds on (most cited first, 23 of 23)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [A Distributional Perspective on Reinforcement Learning](https://mlsanity.com/paper/1707.06887/) (2017)
- [dm_control: Software and Tasks for Continuous Control](https://mlsanity.com/paper/2006.12983/) (2020)
- [Distributed Distributional Deterministic Policy Gradients](https://mlsanity.com/paper/1804.08617/) (2018)
- [Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots](https://mlsanity.com/paper/2103.14295/) (2021)
- [MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policies](https://mlsanity.com/paper/1905.09808/) (2019)
- [Neural probabilistic motor primitives for humanoid control](https://mlsanity.com/paper/1811.11711/) (2018)
- [Time Limits in Reinforcement Learning](https://mlsanity.com/paper/1712.00378/) (2017)
- [Learning human behaviors from motion capture by adversarial imitation](https://mlsanity.com/paper/1707.02201/) (2017)
- [Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?](https://mlsanity.com/paper/1611.01055/) (2016)
- [Feedback Control For Cassie With Deep Reinforcement Learning](https://mlsanity.com/paper/1803.05580/) (2018)
- [Synthesis of Biologically Realistic Human Motion Using Joint Torque Actuation](https://mlsanity.com/paper/1904.13041/) (2019)
- [Tonic: A Deep Reinforcement Learning Library for Fast Prototyping and Benchmarking](https://mlsanity.com/paper/2011.07537/) (2020)
- [Discovering Diverse Athletic Jumping Strategies](https://mlsanity.com/paper/2105.00371/) (2021)
- [Efficient and Robust Reinforcement Learning with Uncertainty-based Value Expansion](https://mlsanity.com/paper/1912.05328/) (2019)
- [Animated Cassie: A Dynamic Relatable Robotic Character](https://mlsanity.com/paper/2009.02846/) (2020)
- [Interactive Modelling of Volumetric Musculoskeletal Anatomy](https://mlsanity.com/paper/2106.05161/) (2021)
- [Dynamic Walking with Compliance on a Cassie Bipedal Robot](https://mlsanity.com/paper/1904.11104/) (2019)
- [EMU: Efficient Muscle Simulation In Deformation Space](https://mlsanity.com/paper/2006.08821/) (2020)
- [Run, skeleton, run: skeletal model in a physics-based simulation](https://mlsanity.com/paper/1711.06922/) (2017)
- [Distributed Soft Actor-Critic with Multivariate Reward Representation and Knowledge Distillation](https://mlsanity.com/paper/1911.13056/) (2019)
- [Sample Efficient Ensemble Learning with Catalyst.RL](https://mlsanity.com/paper/2003.14210/) (2020)

## Built on by (most cited first, 10 of 10)

- [MyoSuite -- A contact-rich simulation suite for musculoskeletal motor control](https://mlsanity.com/paper/2205.13600/) (2022)
- [DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systems](https://mlsanity.com/paper/2206.00484/) (2022)
- [Self Model for Embodied Intelligence: Modeling Full-Body Human Musculoskeletal System and Locomotion Control with Hierarchical Low-Dimensional Representation](https://mlsanity.com/paper/2312.05473/) (2023)
- [DynSyn: Dynamical Synergistic Representation for Efficient Learning and Control in Overactuated Embodied Systems](https://mlsanity.com/paper/2407.11472/) (2024)
- [Learning with Muscles: Benefits for Data-Efficiency and Robustness in Anthropomorphic Tasks](https://mlsanity.com/paper/2207.03952/) (2022)
- [Scalable Exploration for High-Dimensional Continuous Control via Value-Guided Flow](https://mlsanity.com/paper/2601.19707/) (2026)
- [Arnold: A multi-task, multi-embodiment muscle transformer policy](https://mlsanity.com/paper/2508.18066/) (2025)
- [Musculoskeletal simulation of limb movement biomechanics in Drosophila melanogaster](https://mlsanity.com/paper/2509.06426/) (2025)
- [Scaling Whole-Body Human Musculoskeletal Behavior Emulation for Specificity and Diversity](https://mlsanity.com/paper/2603.29332/) (2026)
- [Embodied sensorimotor control: computational modeling of the neural control of movement](https://mlsanity.com/paper/2509.14360/) (2025)

## Abstract (arXiv)

Muscle-actuated control is a research topic that spans multiple domains, including biomechanics, neuroscience, reinforcement learning, robotics, and graphics. This type of control is particularly challenging as bodies are often overactuated and dynamics are delayed and non-linear. It is however a very well tested and tuned actuation mechanism that has undergone millions of years of evolution with interesting properties exploiting passive forces and efficient energy storage of muscle-tendon units. To facilitate research on muscle-actuated simulation, we release a 3D musculoskeletal simulation of an ostrich based on the MuJoCo physics engine. The ostrich is one of the fastest bipeds on earth and therefore makes an excellent model for studying muscle-actuated bipedal locomotion. The model is based on CT scans and dissections used to collect actual muscle data, such as insertion sites, lengths, and pennation angles. Along with this model, we also provide a set of reinforcement learning tasks, including reference motion tracking, running, and neck control, used to infer muscle actuation patterns. The reference motion data is based on motion capture clips of various behaviors that we preprocessed and adapted to our model. This paper describes how the model was built and iteratively improved using the tasks. We also evaluate the accuracy of the muscle actuation patterns by comparing them to experimentally collected electromyographic data from locomoting birds. The results demonstrate the need for rich reward signals or regularization techniques to constrain muscle excitations and produce realistic movements. Overall, we believe that this work can provide a useful bridge between fields of research interested in muscle actuation.