# Sophisticated Policies from Epistemic Priors

arXiv: https://arxiv.org/abs/2607.19518 · HTML: https://mlsanity.com/paper/2607.19518/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 29 references that are in the map.
- First posted on arXiv 21 Jul 2026.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [Active inference on discrete state-spaces: a synthesis](https://mlsanity.com/paper/2001.07203/) (2020)
- [The free energy principle for action and perception: A mathematical review](https://mlsanity.com/paper/1705.09156/) (2017)
- [Sophisticated Inference](https://mlsanity.com/paper/2006.04120/) (2020)
- [pymdp: A Python library for active inference in discrete state spaces](https://mlsanity.com/paper/2201.03904/) (2022)
- [Branching Time Active Inference: the theory and its generality](https://mlsanity.com/paper/2111.11107/) (2021)
- [On efficient computation in active inference](https://mlsanity.com/paper/2307.00504/) (2023)
- [Active Inference as a Model of Agency](https://mlsanity.com/paper/2401.12917/) (2024)
- [Expected Free Energy-based Planning as Variational Inference](https://mlsanity.com/paper/2504.14898/) (2025)
- [What type of inference is planning?](https://mlsanity.com/paper/2406.17863/) (2024)
- [Realising Synthetic Active Inference Agents, Part II: Variational Message Updates](https://mlsanity.com/paper/2306.02733/) (2023)
- [A Message Passing Realization of Expected Free Energy Minimization](https://mlsanity.com/paper/2508.02197/) (2025)
- [Realising Synthetic Active Inference Agents, Part I: Epistemic Objectives and Graphical Specification Language](https://mlsanity.com/paper/2306.08014/) (2023)
- [On Predictive planning and counterfactual learning in active inference](https://mlsanity.com/paper/2403.12417/) (2024)

## Abstract (arXiv)

Sophisticated Inference is a variant of active inference often associated with recursive belief modeling and tree search. We argue that its central computational role is simpler: within a planning horizon, it makes active inference closed-loop by allowing future actions to depend on future states and observations. This closed-loop structure can be represented in the epistemic-prior variational free energy framework. Epistemic priors supply the active-inference objective, while a joint posterior over future states and actions supplies the state-contingent control structure. We evaluate this decomposition in the Reactivity Maze, a stochastic benchmark designed to separate epistemic incentive from inner-horizon closed-loop control. The comparison includes three variational objectives with the same state-action posterior family, an action-state factorized active inference objective, Sophisticated Inference, and standard Expected Free Energy planning. The results show that neither ingredient is sufficient on its own. Methods without an epistemic component do not seek information, while methods that prevent future actions from depending on future states cannot turn information into reliable goal-reaching. By contrast, both Sophisticated Inference and full-joint epistemic-prior active inference solve the environment by combining epistemic drive with closed-loop inference. These results show that the advantage associated with Sophisticated Inference need not be specific to tree search itself. It arises from the closed-loop form of active inference, and this form can be represented in epistemic-prior variational inference when the posterior keeps future actions dependent on future states.