# Black-Box Data-efficient Policy Search for Robotics

arXiv: https://arxiv.org/abs/1703.07261 · HTML: https://mlsanity.com/paper/1703.07261/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Mar 2017.

## Built on by (most cited first, 14 of 14)

- [Robot Learning from Randomized Simulations: A Review](https://mlsanity.com/paper/2111.00956/) (2021)
- [Model-Based Policy Search Using Monte Carlo Gradient Estimation with Real Systems Application](https://mlsanity.com/paper/2101.12115/) (2021)
- [SafeAPT: Safe Simulation-to-Real Robot Learning using Diverse Policies Learned in Simulation](https://mlsanity.com/paper/2201.13248/) (2022)
- [Skill-based Multi-objective Reinforcement Learning of Industrial Robot Tasks with Planning and Knowledge Integration](https://mlsanity.com/paper/2203.10033/) (2022)
- [Learning of Parameters in Behavior Trees for Movement Skills](https://mlsanity.com/paper/2109.13050/) (2021)
- [Learning Control from Raw Position Measurements](https://mlsanity.com/paper/2301.13183/) (2023)
- [Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Review](https://mlsanity.com/paper/2404.17687/) (2024)
- [Controlled Gaussian Process Dynamical Models with Application to Robotic Cloth Manipulation](https://mlsanity.com/paper/2103.06615/) (2021)
- [Regret-Aware Black-Box Optimization with Natural Gradients, Trust-Regions and Entropy Control](https://mlsanity.com/paper/2206.06090/) (2022)
- [Adaptable Recovery Behaviors in Robotics: A Behavior Trees and Motion Generators(BTMG) Approach for Failure Management](https://mlsanity.com/paper/2404.06129/) (2024)
- [Model-based Policy Search for Partially Measurable Systems](https://mlsanity.com/paper/2101.08740/) (2021)
- [GPU-Accelerated Policy Optimization via Batch Automatic Differentiation of Gaussian Processes for Real-World Control](https://mlsanity.com/paper/2202.13638/) (2022)
- [FORESEE: Prediction with Expansion-Compression Unscented Transform for Online Policy Optimization](https://mlsanity.com/paper/2209.12644/) (2022)
- [Accelerating Model-Based Reinforcement Learning using Non-Linear Trajectory Optimization](https://mlsanity.com/paper/2506.02767/) (2025)

## Abstract (arXiv)

The most data-efficient algorithms for reinforcement learning (RL) in robotics are based on uncertain dynamical models: after each episode, they first learn a dynamical model of the robot, then they use an optimization algorithm to find a policy that maximizes the expected return given the model and its uncertainties. It is often believed that this optimization can be tractable only if analytical, gradient-based algorithms are used; however, these algorithms require using specific families of reward functions and policies, which greatly limits the flexibility of the overall approach. In this paper, we introduce a novel model-based RL algorithm, called Black-DROPS (Black-box Data-efficient RObot Policy Search) that: (1) does not impose any constraint on the reward function or the policy (they are treated as black-boxes), (2) is as data-efficient as the state-of-the-art algorithm for data-efficient RL in robotics, and (3) is as fast (or faster) than analytical approaches when several cores are available. The key idea is to replace the gradient-based optimization algorithm with a parallel, black-box algorithm that takes into account the model uncertainties. We demonstrate the performance of our new algorithm on two standard control benchmark problems (in simulation) and a low-cost robotic manipulator (with a real robot).