# Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics

arXiv: https://arxiv.org/abs/1709.06917 · HTML: https://mlsanity.com/paper/1709.06917/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2017.

## Built on by (most cited first, 6 of 6)

- [Robot Learning from Randomized Simulations: A Review](https://mlsanity.com/paper/2111.00956/) (2021)
- [Learning of Parameters in Behavior Trees for Movement Skills](https://mlsanity.com/paper/2109.13050/) (2021)
- [A Simple Approach to Continual Learning by Transferring Skill Parameters](https://mlsanity.com/paper/2110.10255/) (2021)
- [Controlled Gaussian Process Dynamical Models with Application to Robotic Cloth Manipulation](https://mlsanity.com/paper/2103.06615/) (2021)
- [RegHEC: Hand-Eye Calibration via Simultaneous Multi-view Point Clouds Registration of Arbitrary Object](https://mlsanity.com/paper/2304.14092/) (2023)
- [First do not fall: learning to exploit a wall with a damaged humanoid robot](https://mlsanity.com/paper/2203.00316/) (2022)

## Abstract (arXiv)

The most data-efficient algorithms for reinforcement learning in robotics are model-based policy search algorithms, which alternate between learning a dynamical model of the robot and optimizing a policy to maximize the expected return given the model and its uncertainties. Among the few proposed approaches, the recently introduced Black-DROPS algorithm exploits a black-box optimization algorithm to achieve both high data-efficiency and good computation times when several cores are used; nevertheless, like all model-based policy search approaches, Black-DROPS does not scale to high dimensional state/action spaces. In this paper, we introduce a new model learning procedure in Black-DROPS that leverages parameterized black-box priors to (1) scale up to high-dimensional systems, and (2) be robust to large inaccuracies of the prior information. We demonstrate the effectiveness of our approach with the "pendubot" swing-up task in simulation and with a physical hexapod robot (48D state space, 18D action space) that has to walk forward as fast as possible. The results show that our new algorithm is more data-efficient than previous model-based policy search algorithms (with and without priors) and that it can allow a physical 6-legged robot to learn new gaits in only 16 to 30 seconds of interaction time.