# What type of inference is planning?

arXiv: https://arxiv.org/abs/2406.17863 · HTML: https://mlsanity.com/paper/2406.17863/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 52 references that are in the map.
- First posted on arXiv 25 Jun 2024.

## Builds on (most cited first, 7 of 7)

- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [Risk-sensitive Reinforcement Learning](https://mlsanity.com/paper/1311.2097/) (2013)
- [Causal Consistency of Structural Equation Models](https://mlsanity.com/paper/1707.00819/) (2017)
- [Linear Programming for Large-Scale Markov Decision Problems](https://mlsanity.com/paper/1402.6763/) (2014)
- [Variational Algorithms for Marginal MAP](https://mlsanity.com/paper/1302.6584/) (2013)
- [Approximate Inference for Stochastic Planning in Factored Spaces](https://mlsanity.com/paper/2203.12139/) (2022)
- [Finite-Horizon Constrained MDPs With Both Additive And Multiplicative Utilities](https://mlsanity.com/paper/2303.07834/) (2023)

## Built on by (most cited first, 9 of 9)

- [Large Language Models for Planning: A Comprehensive and Systematic Survey](https://mlsanity.com/paper/2505.19683/) (2025)
- [Soft Policy Optimization: Online Off-Policy RL for Sequence Models](https://mlsanity.com/paper/2503.05453/) (2025)
- [Expected Free Energy-based Planning as Variational Inference](https://mlsanity.com/paper/2504.14898/) (2025)
- [A Message Passing Realization of Expected Free Energy Minimization](https://mlsanity.com/paper/2508.02197/) (2025)
- [What Type of Inference is Active Inference?](https://mlsanity.com/paper/2606.04935/) (2026)
- [Active Inference as a Convex Markov Decision Process](https://mlsanity.com/paper/2607.20152/) (2026)
- [Active Inference is a Subtype of Variational Inference](https://mlsanity.com/paper/2511.18955/) (2025)
- [Expected Free Energy-based Planning as Variational Inference](https://mlsanity.com/paper/2606.20658/) (2026)
- [Sophisticated Policies from Epistemic Priors](https://mlsanity.com/paper/2607.19518/) (2026)

## Abstract (arXiv)

Multiple types of inference are available for probabilistic graphical models, e.g., marginal, maximum-a-posteriori, and even marginal maximum-a-posteriori. Which one do researchers mean when they talk about "planning as inference"? There is no consistency in the literature, different types are used, and their ability to do planning is further entangled with specific approximations or additional constraints. In this work we use the variational framework to show that, just like all commonly used types of inference correspond to different weightings of the entropy terms in the variational problem, planning corresponds exactly to a different set of weights. This means that all the tricks of variational inference are readily applicable to planning. We develop an analogue of loopy belief propagation that allows us to perform approximate planning in factored-state Markov decisions processes without incurring intractability due to the exponentially large state space. The variational perspective shows that the previous types of inference for planning are only adequate in environments with low stochasticity, and allows us to characterize each type by its own merits, disentangling the type of inference from the additional approximations that its practical use requires. We validate these results empirically on synthetic MDPs and tasks posed in the International Planning Competition.