# Deriving time-averaged active inference from control principles

arXiv: https://arxiv.org/abs/2208.10601 · HTML: https://mlsanity.com/paper/2208.10601/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 62 references that are in the map.
- First posted on arXiv 22 Aug 2022.

## Builds on (most cited first, 11 of 11)

- [Planning with Goal-Conditioned Policies](https://mlsanity.com/paper/1911.08453/) (2019)
- [Active inference on discrete state-spaces: a synthesis](https://mlsanity.com/paper/2001.07203/) (2020)
- [Active Inference in Robotics and Artificial Agents: Survey and Challenges](https://mlsanity.com/paper/2112.01871/) (2021)
- [On-Policy Deep Reinforcement Learning for the Average-Reward Criterion](https://mlsanity.com/paper/2106.07329/) (2021)
- [Online Learning for Unknown Partially Observable MDPs](https://mlsanity.com/paper/2102.12661/) (2021)
- [Path Integral Control and State Dependent Feedback](https://mlsanity.com/paper/1406.4026/) (2014)
- [Hindsight Expectation Maximization for Goal-conditioned Reinforcement Learning](https://mlsanity.com/paper/2006.07549/) (2020)
- [DisCo RL: Distribution-Conditioned Reinforcement Learning for General-Purpose Policies](https://mlsanity.com/paper/2104.11707/) (2021)
- [Goal Kernel Planning: Linearly-Solvable Non-Markovian Policies for Logical Tasks with Goal-Conditioned Options](https://mlsanity.com/paper/2007.02527/) (2020)
- [Nonparametric Infinite Horizon Kullback-Leibler Stochastic Control](https://mlsanity.com/paper/1404.3984/) (2014)
- [Formal models of memory based on temporally-varying representations](https://mlsanity.com/paper/2201.01796/) (2022)

## Built on by (most cited first, 1 of 1)

- [Realising Synthetic Active Inference Agents, Part II: Variational Message Updates](https://mlsanity.com/paper/2306.02733/) (2023)

## Abstract (arXiv)

Active inference offers a principled account of behavior as minimizing average sensory surprise over time. Applications of active inference to control problems have heretofore tended to focus on finite-horizon or discounted-surprise problems, despite deriving from the infinite-horizon, average-surprise imperative of the free-energy principle. Here we derive an infinite-horizon, average-surprise formulation of active inference from optimal control principles. Our formulation returns to the roots of active inference in neuroanatomy and neurophysiology, formally reconnecting active inference to optimal feedback control. Our formulation provides a unified objective functional for sensorimotor control and allows for reference states to vary over time.