# Active Inference and Epistemic Value in Graphical Models

arXiv: https://arxiv.org/abs/2109.00541 · HTML: https://mlsanity.com/paper/2109.00541/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 54 references that are in the map.
- First posted on arXiv 1 Sep 2021.

## Builds on (most cited first, 13 of 13)

- [Active inference on discrete state-spaces: a synthesis](https://mlsanity.com/paper/2001.07203/) (2020)
- [Predictive Coding Approximates Backprop along Arbitrary Computation Graphs](https://mlsanity.com/paper/2006.04182/) (2020)
- [Sophisticated Inference](https://mlsanity.com/paper/2006.04120/) (2020)
- [Deep Active Inference as Variational Policy Gradients](https://mlsanity.com/paper/1907.03876/) (2019)
- [Reinforcement Learning through Active Inference](https://mlsanity.com/paper/2002.12636/) (2020)
- [Deep Active Inference](https://mlsanity.com/paper/1709.02341/) (2017)
- [Whence the Expected Free Energy?](https://mlsanity.com/paper/2004.08128/) (2020)
- [Action and Perception as Divergence Minimization](https://mlsanity.com/paper/2009.01791/) (2020)
- [Scaling active inference](https://mlsanity.com/paper/1911.10601/) (2019)
- [A Factor Graph Approach to Automated Design of Bayesian Signal Processing Algorithms](https://mlsanity.com/paper/1811.03407/) (2018)
- [Understanding the Origin of Information-Seeking Exploration in Probabilistic Objectives for Control](https://mlsanity.com/paper/2103.06859/) (2021)
- [Bayesian brains and the R'enyi divergence](https://mlsanity.com/paper/2107.05438/) (2021)
- [Chance-Constrained Active Inference](https://mlsanity.com/paper/2102.08792/) (2021)

## Built on by (most cited first, 6 of 6)

- [Realising Synthetic Active Inference Agents, Part II: Variational Message Updates](https://mlsanity.com/paper/2306.02733/) (2023)
- [Information-seeking polynomial NARX model-predictive control through expected free energy minimization](https://mlsanity.com/paper/2312.15046/) (2023)
- [Coupled autoregressive active inference agents for control of multi-joint dynamical systems](https://mlsanity.com/paper/2410.10415/) (2024)
- [Message passing-based inference in an autoregressive active inference agent](https://mlsanity.com/paper/2509.25482/) (2025)
- [Planning to avoid ambiguous states through Gaussian approximations to non-linear sensors in active inference agents](https://mlsanity.com/paper/2409.01974/) (2024)
- [Expected free energy as an information constraint on the Bethe Lagrangian](https://mlsanity.com/paper/2608.17167/) (2026)

## Abstract (arXiv)

The Free Energy Principle (FEP) postulates that biological agents perceive and interact with their environment in order to minimize a Variational Free Energy (VFE) with respect to a generative model of their environment. The inference of a policy (future control sequence) according to the FEP is known as Active Inference (AIF). The AIF literature describes multiple VFE objectives for policy planning that lead to epistemic (information-seeking) behavior. However, most objectives have limited modeling flexibility. This paper approaches epistemic behavior from a constrained Bethe Free Energy (CBFE) perspective. Crucially, variational optimization of the CBFE can be expressed in terms of message passing on free-form generative models. The key intuition behind the CBFE is that we impose a point-mass constraint on predicted outcomes, which explicitly encodes the assumption that the agent will make observations in the future. We interpret the CBFE objective in terms of its constituent behavioral drives. We then illustrate resulting behavior of the CBFE by planning and interacting with a simulated T-maze environment. Simulations for the T-maze task illustrate how the CBFE agent exhibits an epistemic drive, and actively plans ahead to account for the impact of predicted outcomes. Compared to an EFE agent, the CBFE agent incurs expected reward in significantly more environmental scenarios. We conclude that CBFE optimization by message passing suggests a general mechanism for epistemic-aware AIF in free-form generative models.