# Model-free Reinforcement Learning for Branching Markov Decision Processes

arXiv: https://arxiv.org/abs/2106.06777 · HTML: https://mlsanity.com/paper/2106.06777/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 35 references that are in the map.
- First posted on arXiv 12 Jun 2021.

## Builds on (most cited first, 1 of 1)

- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)

## Built on by (most cited first, 1 of 1)

- [Recursive Reinforcement Learning](https://mlsanity.com/paper/2206.11430/) (2022)

## Abstract (arXiv)

We study reinforcement learning for the optimal control of Branching Markov Decision Processes (BMDPs), a natural extension of (multitype) Branching Markov Chains (BMCs). The state of a (discrete-time) BMCs is a collection of entities of various types that, while spawning other entities, generate a payoff. In comparison with BMCs, where the evolution of a each entity of the same type follows the same probabilistic pattern, BMDPs allow an external controller to pick from a range of options. This permits us to study the best/worst behaviour of the system. We generalise model-free reinforcement learning techniques to compute an optimal control strategy of an unknown BMDP in the limit. We present results of an implementation that demonstrate the practicality of the approach.