# Branching Time Active Inference: the theory and its generality

arXiv: https://arxiv.org/abs/2111.11107 · HTML: https://mlsanity.com/paper/2111.11107/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 57 references that are in the map.
- First posted on arXiv 22 Nov 2021.

## Builds on (most cited first, 16 of 16)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Playing FPS Games with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05521/) (2016)
- [Active inference on discrete state-spaces: a synthesis](https://mlsanity.com/paper/2001.07203/) (2020)
- [Deep active inference agents using Monte-Carlo methods](https://mlsanity.com/paper/2006.04176/) (2020)
- [Sophisticated Inference](https://mlsanity.com/paper/2006.04120/) (2020)
- [Deep Active Inference as Variational Policy Gradients](https://mlsanity.com/paper/1907.03876/) (2019)
- [Whence the Expected Free Energy?](https://mlsanity.com/paper/2004.08128/) (2020)
- [Action and Perception as Divergence Minimization](https://mlsanity.com/paper/2009.01791/) (2020)
- [End-to-End Pixel-Based Deep Active Inference for Body Perception and Action](https://mlsanity.com/paper/2001.05847/) (2019)
- [Active Inference and Behavior Trees for Reactive Action Planning and Execution in Robotics](https://mlsanity.com/paper/2011.09756/) (2020)
- [Learning Perception and Planning with Deep Active Inference](https://mlsanity.com/paper/2001.11841/) (2020)
- [A Factor Graph Approach to Automated Design of Bayesian Signal Processing Algorithms](https://mlsanity.com/paper/1811.03407/) (2018)

## Built on by (most cited first, 15 of 15)

- [On efficient computation in active inference](https://mlsanity.com/paper/2307.00504/) (2023)
- [Active Inference as a Model of Agency](https://mlsanity.com/paper/2401.12917/) (2024)
- [Expected Free Energy-based Planning as Variational Inference](https://mlsanity.com/paper/2504.14898/) (2025)
- [Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents](https://mlsanity.com/paper/2203.10592/) (2022)
- [Deconstructing deep active inference](https://mlsanity.com/paper/2303.01618/) (2023)
- [A Message Passing Realization of Expected Free Energy Minimization](https://mlsanity.com/paper/2508.02197/) (2025)
- [Realising Synthetic Active Inference Agents, Part I: Epistemic Objectives and Graphical Specification Language](https://mlsanity.com/paper/2306.08014/) (2023)
- [Branching Time Active Inference with Bayesian Filtering](https://mlsanity.com/paper/2112.07406/) (2021)
- [Multi-Modal and Multi-Factor Branching Time Active Inference](https://mlsanity.com/paper/2206.12503/) (2022)
- [What Type of Inference is Active Inference?](https://mlsanity.com/paper/2606.04935/) (2026)
- [Efficient search of active inference policy spaces using k-means](https://mlsanity.com/paper/2209.02550/) (2022)
- [Boosting MCTS with Free Energy Minimization](https://mlsanity.com/paper/2501.13083/) (2025)
- [A memory-based active inference model of DishBrain-like adaptive behaviour](https://mlsanity.com/paper/2508.06980/) (2025)
- [Expected Free Energy-based Planning as Variational Inference](https://mlsanity.com/paper/2606.20658/) (2026)
- [Sophisticated Policies from Epistemic Priors](https://mlsanity.com/paper/2607.19518/) (2026)

## Abstract (arXiv)

Over the last 10 to 15 years, active inference has helped to explain various brain mechanisms from habit formation to dopaminergic discharge and even modelling curiosity. However, the current implementations suffer from an exponential (space and time) complexity class when computing the prior over all the possible policies up to the time-horizon. Fountas et al (2020) used Monte Carlo tree search to address this problem, leading to impressive results in two different tasks. In this paper, we present an alternative framework that aims to unify tree search and active inference by casting planning as a structure learning problem. Two tree search algorithms are then presented. The first propagates the expected free energy forward in time (i.e., towards the leaves), while the second propagates it backward (i.e., towards the root). Then, we demonstrate that forward and backward propagations are related to active inference and sophisticated inference, respectively, thereby clarifying the differences between those two planning strategies.