# On the Fly Adaptation of Behavior Tree-Based Policies through Reinforcement Learning

arXiv: https://arxiv.org/abs/2503.06309 · HTML: https://mlsanity.com/paper/2503.06309/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 28 references that are in the map.
- First posted on arXiv 8 Mar 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 18 of 18)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Contextual Markov Decision Processes](https://mlsanity.com/paper/1502.02259/) (2015)
- [Multi-Task Reinforcement Learning with Context-based Representations](https://mlsanity.com/paper/2102.06177/) (2021)
- [Behavior Trees in Robotics and AI: An Introduction](https://mlsanity.com/paper/1709.00084/) (2017)
- [Markov Decision Processes with Continuous Side Information](https://mlsanity.com/paper/1711.05726/) (2017)
- [A Survey of Behavior Trees in Robotics and AI](https://mlsanity.com/paper/2005.05842/) (2020)
- [Contextualize Me -- The Case for Context in Reinforcement Learning](https://mlsanity.com/paper/2202.04500/) (2022)
- [Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environment](https://mlsanity.com/paper/2104.05632/) (2021)
- [A survey on policy search algorithms for learning robot controllers in a handful of trials](https://mlsanity.com/paper/1807.02303/) (2018)
- [Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity](https://mlsanity.com/paper/2211.01500/) (2022)
- [Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policies](https://mlsanity.com/paper/2310.16686/) (2023)
- [Sampling-based Planning of In-Hand Manipulation with External Pushes](https://mlsanity.com/paper/1707.00318/) (2017)
- [BeBOP -- Combining Reactive Planning and Bayesian Optimization to Solve Robotic Manipulation Tasks](https://mlsanity.com/paper/2310.00971/) (2023)
- [Learning to Adapt the Parameters of Behavior Trees and Motion Generators (BTMGs) to Task Variations](https://mlsanity.com/paper/2303.08209/) (2023)
- [Learning Extrinsic Dexterity with Parameterized Manipulation Primitives](https://mlsanity.com/paper/2310.17785/) (2023)
- [Inferring Behavior-Specific Context Improves Zero-Shot Generalization in Reinforcement Learning](https://mlsanity.com/paper/2404.09521/) (2024)
- [Learning of Parameters in Behavior Trees for Movement Skills](https://mlsanity.com/paper/2109.13050/) (2021)
- [A Stack-of-Tasks Approach Combined with Behavior Trees: a New Framework for Robot Control](https://mlsanity.com/paper/2209.08619/) (2022)

## Built on by (most cited first, 1 of 1)

- [SharedKV-BT: Node-Local Typed Decisions for Behavior-Tree Agents](https://mlsanity.com/paper/2610.07327/) (2026)

## Abstract (arXiv)

With the rising demand for flexible manufacturing, robots are increasingly expected to operate in dynamic environments where local -- such as slight offsets or size differences in workpieces -- are common. We propose to address the problem of adapting robot behaviors to these task variations with a sample-efficient hierarchical reinforcement learning approach adapting Behavior Tree (BT)-based policies. We maintain the core BT properties as an interpretable, modular framework for structuring reactive behaviors, but extend their use beyond static tasks by inherently accommodating local task variations. To show the efficiency and effectiveness of our approach, we conduct experiments both in simulation and on a Franka Emika Panda 7-DoF, with the manipulator adapting to different obstacle avoidance and pivoting tasks.