# Reinforcement Learning through Active Inference

arXiv: https://arxiv.org/abs/2002.12636 · HTML: https://mlsanity.com/paper/2002.12636/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2020.

## Built on by (most cited first, 30 of 47)

- [Predictive Coding: a Theoretical and Experimental Review](https://mlsanity.com/paper/2107.12979/) (2021)
- [Active Inference in Robotics and Artificial Agents: Survey and Challenges](https://mlsanity.com/paper/2112.01871/) (2021)
- [pymdp: A Python library for active inference in discrete state spaces](https://mlsanity.com/paper/2201.03904/) (2022)
- [At the Dawn of Generative AI Era: A Tutorial-cum-Survey on New Frontiers in 6G Wireless Intelligence](https://mlsanity.com/paper/2402.18587/) (2024)
- [The Free Energy Principle for Perception and Action: A Deep Learning Perspective](https://mlsanity.com/paper/2207.06415/) (2022)
- [Contrastive Active Inference](https://mlsanity.com/paper/2110.10083/) (2021)
- [Backprop-Free Reinforcement Learning with Active Neural Generative Coding](https://mlsanity.com/paper/2107.07046/) (2021)
- [Understanding the Origin of Information-Seeking Exploration in Probabilistic Objectives for Control](https://mlsanity.com/paper/2103.06859/) (2021)
- [Active Exploration for Robotic Manipulation](https://mlsanity.com/paper/2210.12806/) (2022)
- [R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Models](https://mlsanity.com/paper/2409.14216/) (2024)
- [Applications of the Free Energy Principle to Machine Learning and Neuroscience](https://mlsanity.com/paper/2107.00140/) (2021)
- [Prior Preference Learning from Experts:Designing a Reward with Active Inference](https://mlsanity.com/paper/2101.08937/) (2021)
- [Exploration via Planning for Information about the Optimal Trajectory](https://mlsanity.com/paper/2210.04642/) (2022)
- [Complex behavior from intrinsic motivation to occupy action-state path space](https://mlsanity.com/paper/2205.10316/) (2022)
- [Explore the Context: Optimal Data Collection for Context-Conditional Dynamics Models](https://mlsanity.com/paper/2102.11394/) (2021)
- [Active Inference and Epistemic Value in Graphical Models](https://mlsanity.com/paper/2109.00541/) (2021)
- [Adaptive Stream Processing on Edge Devices through Active Inference](https://mlsanity.com/paper/2409.17937/) (2024)
- [Chance-Constrained Active Inference](https://mlsanity.com/paper/2102.08792/) (2021)
- [Online reinforcement learning with sparse rewards through an active inference capsule](https://mlsanity.com/paper/2106.02390/) (2021)
- [Successor Representation Active Inference](https://mlsanity.com/paper/2207.09897/) (2022)
- [Active Inference in String Diagrams: A Categorical Account of Predictive Processing and Free Energy](https://mlsanity.com/paper/2308.00861/) (2023)
- [Intrinsic Rewards for Exploration without Harm from Observational Noise: A Simulation Study Based on the Free Energy Principle](https://mlsanity.com/paper/2405.07473/) (2024)
- [Active Inference Tree Search in Large POMDPs](https://mlsanity.com/paper/2103.13860/) (2021)
- [Deep Active Inference for Pixel-Based Discrete Control: Evaluation on the Car Racing Problem](https://mlsanity.com/paper/2109.04155/) (2021)
- [Active Inference for Robotic Manipulation](https://mlsanity.com/paper/2206.10313/) (2022)
- [Sophisticated Learning: A novel algorithm for active learning during model-based planning](https://mlsanity.com/paper/2308.08029/) (2023)
- [Free Energy Projective Simulation (FEPS): Active inference with interpretability](https://mlsanity.com/paper/2411.14991/) (2024)
- [The Missing Reward: Active Inference in the Era of Experience](https://mlsanity.com/paper/2508.05619/) (2025)
- [A Mathematical Walkthrough and Discussion of the Free Energy Principle](https://mlsanity.com/paper/2108.13343/) (2021)
- [Life-inspired Interoceptive Artificial Intelligence for Autonomous and Adaptive Agents](https://mlsanity.com/paper/2309.05999/) (2023)

## Abstract (arXiv)

The central tenet of reinforcement learning (RL) is that agents seek to maximize the sum of cumulative rewards. In contrast, active inference, an emerging framework within cognitive and computational neuroscience, proposes that agents act to maximize the evidence for a biased generative model. Here, we illustrate how ideas from active inference can augment traditional RL approaches by (i) furnishing an inherent balance of exploration and exploitation, and (ii) providing a more flexible conceptualization of reward. Inspired by active inference, we develop and implement a novel objective for decision making, which we term the free energy of the expected future. We demonstrate that the resulting algorithm successfully balances exploration and exploitation, simultaneously achieving robust performance on several challenging RL benchmarks with sparse, well-shaped, and no rewards.