# Efficient Hyperparameter Optimization for Physics-based Character Animation

arXiv: https://arxiv.org/abs/2104.12365 · HTML: https://mlsanity.com/paper/2104.12365/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 91 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 30 of 30)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Scalable Bayesian Optimization Using Deep Neural Networks](https://mlsanity.com/paper/1502.05700/) (2015)
- [Emergence of Locomotion Behaviours in Rich Environments](https://mlsanity.com/paper/1707.02286/) (2017)
- [DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills](https://mlsanity.com/paper/1804.02717/) (2018)
- [Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets](https://mlsanity.com/paper/1605.07079/) (2016)
- [Multi-fidelity Bayesian Optimisation with Continuous Approximations](https://mlsanity.com/paper/1703.06240/) (2017)
- [One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control](https://mlsanity.com/paper/2007.04976/) (2020)
- [Freeze-Thaw Bayesian Optimization](https://mlsanity.com/paper/1406.3896/) (2014)
- [Learning Symmetric and Low-energy Locomotion](https://mlsanity.com/paper/1801.08093/) (2018)
- [Reinforcement Learning for Improving Agent Design](https://mlsanity.com/paper/1810.03779/) (2018)
- [Sim-to-Real Transfer for Biped Locomotion](https://mlsanity.com/paper/1903.01390/) (2019)
- [Jointly Learning to Construct and Control Agents using Deep Reinforcement Learning](https://mlsanity.com/paper/1801.01432/) (2018)
- [ALLSTEPS: Curriculum-driven Learning of Stepping Stone Skills](https://mlsanity.com/paper/2005.04323/) (2020)
- [A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes](https://mlsanity.com/paper/1811.00755/) (2018)
- [Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning](https://mlsanity.com/paper/1911.06832/) (2019)
- [Data-efficient Learning of Morphology and Controller for a Microrobot](https://mlsanity.com/paper/1905.01334/) (2019)
- [Sequential Gallery for Interactive Visual Design Optimization](https://mlsanity.com/paper/2005.04107/) (2020)
- [Bayesian Optimization Meets Riemannian Manifolds in Robot Learning](https://mlsanity.com/paper/1910.04998/) (2019)
- [Multi-fidelity Bayesian Optimization with Max-value Entropy Search and its parallelization](https://mlsanity.com/paper/1901.08275/) (2019)
- [Knowing The What But Not The Where in Bayesian Optimization](https://mlsanity.com/paper/1905.02685/) (2019)
- [Bayesian Optimization for Iterative Learning](https://mlsanity.com/paper/1909.09593/) (2019)
- [Functional Co-Optimization of Articulated Robots](https://mlsanity.com/paper/1707.06617/) (2017)
- [Learning and Exploring Motor Skills with Spacetime Bounds](https://mlsanity.com/paper/2103.16807/) (2021)
- [Neural fidelity warping for efficient robot morphology design](https://mlsanity.com/paper/2012.04195/) (2020)

## Built on by (most cited first, 5 of 5)

- [Composite Motion Learning with Task Control](https://mlsanity.com/paper/2305.03286/) (2023)
- [DiffMimic: Efficient Motion Mimicking with Differentiable Physics](https://mlsanity.com/paper/2304.03274/) (2023)
- [Complex Locomotion Skill Learning via Differentiable Physics](https://mlsanity.com/paper/2206.02341/) (2022)
- [AgentForge: A Flexible Low-Code Platform for Reinforcement Learning Agent Design](https://mlsanity.com/paper/2410.19528/) (2024)
- [SLowRL: Safe Low-Rank Adaptation Reinforcement Learning for Locomotion](https://mlsanity.com/paper/2603.17092/) (2026)

## Abstract (arXiv)

Physics-based character animation has seen significant advances in recent years with the adoption of Deep Reinforcement Learning (DRL). However, DRL-based learning methods are usually computationally expensive and their performance crucially depends on the choice of hyperparameters. Tuning hyperparameters for these methods often requires repetitive training of control policies, which is even more computationally prohibitive. In this work, we propose a novel Curriculum-based Multi-Fidelity Bayesian Optimization framework (CMFBO) for efficient hyperparameter optimization of DRL-based character control systems. Using curriculum-based task difficulty as fidelity criterion, our method improves searching efficiency by gradually pruning search space through evaluation on easier motor skill tasks. We evaluate our method on two physics-based character control tasks: character morphology optimization and hyperparameter tuning of DeepMimic. Our algorithm significantly outperforms state-of-the-art hyperparameter optimization methods applicable for physics-based character animation. In particular, we show that hyperparameters optimized through our algorithm result in at least 5x efficiency gain comparing to author-released settings in DeepMimic.