# Control as Hybrid Inference

arXiv: https://arxiv.org/abs/2007.05838 · HTML: https://mlsanity.com/paper/2007.05838/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jul 2020.

## Built on by (most cited first, 9 of 9)

- [Hybrid Predictive Coding: Inferring, Fast and Slow](https://mlsanity.com/paper/2204.02169/) (2022)
- [Applications of the Free Energy Principle to Machine Learning and Neuroscience](https://mlsanity.com/paper/2107.00140/) (2021)
- [Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents](https://mlsanity.com/paper/2203.10592/) (2022)
- [How does the primate brain combine generative and discriminative computations in vision?](https://mlsanity.com/paper/2401.06005/) (2024)
- [Deep Active Inference Agents for Delayed and Long-Horizon Environments](https://mlsanity.com/paper/2505.19867/) (2025)
- [Successor Representation Active Inference](https://mlsanity.com/paper/2207.09897/) (2022)
- [Learning Policies for Continuous Control via Transition Models](https://mlsanity.com/paper/2209.08033/) (2022)
- [Active Inference Meeting Energy-Efficient Control of Parallel and Identical Machines](https://mlsanity.com/paper/2406.09322/) (2024)
- [When in Doubt, Think Slow: Iterative Reasoning with Latent Imagination](https://mlsanity.com/paper/2402.15283/) (2024)

## Abstract (arXiv)

The field of reinforcement learning can be split into model-based and model-free methods. Here, we unify these approaches by casting model-free policy optimisation as amortised variational inference, and model-based planning as iterative variational inference, within a `control as hybrid inference' (CHI) framework. We present an implementation of CHI which naturally mediates the balance between iterative and amortised inference. Using a didactic experiment, we demonstrate that the proposed algorithm operates in a model-based manner at the onset of learning, before converging to a model-free algorithm once sufficient data have been collected. We verify the scalability of our algorithm on a continuous control benchmark, demonstrating that it outperforms strong model-free and model-based baselines. CHI thus provides a principled framework for harnessing the sample efficiency of model-based planning while retaining the asymptotic performance of model-free policy optimisation.