# Learning of Parameters in Behavior Trees for Movement Skills

arXiv: https://arxiv.org/abs/2109.13050 · HTML: https://mlsanity.com/paper/2109.13050/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 32 references that are in the map.
- First posted on arXiv 27 Sep 2021.

## Builds on (most cited first, 9 of 9)

- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Active Domain Randomization](https://mlsanity.com/paper/1904.04762/) (2019)
- [A Survey of Behavior Trees in Robotics and AI](https://mlsanity.com/paper/2005.05842/) (2020)
- [A survey on policy search algorithms for learning robot controllers in a handful of trials](https://mlsanity.com/paper/1807.02303/) (2018)
- [CoSTAR: Instructing Collaborative Robots with Behavior Trees and Vision](https://mlsanity.com/paper/1611.06145/) (2016)
- [Learning of Behavior Trees for Autonomous Agents](https://mlsanity.com/paper/1504.05811/) (2015)
- [Black-Box Data-efficient Policy Search for Robotics](https://mlsanity.com/paper/1703.07261/) (2017)
- [Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics](https://mlsanity.com/paper/1709.06917/) (2017)
- [Deep Reinforcement Learning to Acquire Navigation Skills for Wheel-Legged Robots in Complex Environments](https://mlsanity.com/paper/1804.10500/) (2018)

## Built on by (most cited first, 4 of 4)

- [Skill-based Multi-objective Reinforcement Learning of Industrial Robot Tasks with Planning and Knowledge Integration](https://mlsanity.com/paper/2203.10033/) (2022)
- [Using Knowledge Representation and Task Planning for Robot-agnostic Skills on the Example of Contact-Rich Wiping Tasks](https://mlsanity.com/paper/2308.14206/) (2023)
- [Adaptable Recovery Behaviors in Robotics: A Behavior Trees and Motion Generators(BTMG) Approach for Failure Management](https://mlsanity.com/paper/2404.06129/) (2024)
- [On the Fly Adaptation of Behavior Tree-Based Policies through Reinforcement Learning](https://mlsanity.com/paper/2503.06309/) (2025)

## Abstract (arXiv)

Reinforcement Learning (RL) is a powerful mathematical framework that allows robots to learn complex skills by trial-and-error. Despite numerous successes in many applications, RL algorithms still require thousands of trials to converge to high-performing policies, can produce dangerous behaviors while learning, and the optimized policies (usually modeled as neural networks) give almost zero explanation when they fail to perform the task. For these reasons, the adoption of RL in industrial settings is not common. Behavior Trees (BTs), on the other hand, can provide a policy representation that a) supports modular and composable skills, b) allows for easy interpretation of the robot actions, and c) provides an advantageous low-dimensional parameter space. In this paper, we present a novel algorithm that can learn the parameters of a BT policy in simulation and then generalize to the physical robot without any additional training. We leverage a physical simulator with a digital twin of our workstation, and optimize the relevant parameters with a black-box optimizer. We showcase the efficacy of our method with a 7-DOF KUKA-iiwa manipulator in a task that includes obstacle avoidance and a contact-rich insertion (peg-in-hole), in which our method outperforms the baselines.