# Object-based active inference

arXiv: https://arxiv.org/abs/2209.01258 · HTML: https://mlsanity.com/paper/2209.01258/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 2 Sep 2022.

## Builds on (most cited first, 12 of 12)

- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [Multi-Object Representation Learning with Iterative Variational Inference](https://mlsanity.com/paper/1903.00450/) (2019)
- [Entity Abstraction in Visual Model-Based Reinforcement Learning](https://mlsanity.com/paper/1910.12827/) (2019)
- [Iterative Amortized Inference](https://mlsanity.com/paper/1807.09356/) (2018)
- [Active inference on discrete state-spaces: a synthesis](https://mlsanity.com/paper/2001.07203/) (2020)
- [COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration](https://mlsanity.com/paper/1905.09275/) (2019)
- [Active Inference in Robotics and Artificial Agents: Survey and Challenges](https://mlsanity.com/paper/2112.01871/) (2021)
- [A free energy principle for a particular physics](https://mlsanity.com/paper/1906.10184/) (2019)
- [Whence the Expected Free Energy?](https://mlsanity.com/paper/2004.08128/) (2020)
- [Capturing the objects of vision with neural networks](https://mlsanity.com/paper/2109.03351/) (2021)
- [Unsupervised Video Decomposition using Spatio-temporal Iterative Inference](https://mlsanity.com/paper/2006.14727/) (2020)

## Built on by (most cited first, 7 of 7)

- [World Models and Predictive Coding for Cognitive and Developmental Robotics: Frontiers and Challenges](https://mlsanity.com/paper/2301.05832/) (2023)
- [Entity-Centric Reinforcement Learning for Object Manipulation from Pixels](https://mlsanity.com/paper/2404.01220/) (2024)
- [Object-Centric World Models from Few-Shot Annotations for Sample-Efficient Reinforcement Learning](https://mlsanity.com/paper/2501.16443/) (2025)
- [Object-Centric Scene Representations using Active Inference](https://mlsanity.com/paper/2302.03288/) (2023)
- [Object-centric proto-symbolic behavioural reasoning from pixels](https://mlsanity.com/paper/2411.17438/) (2024)
- [Deep hybrid models: infer and plan in a dynamic world](https://mlsanity.com/paper/2402.10088/) (2024)
- [Mechanistic Foundations of Goal-Directed Control](https://mlsanity.com/paper/2603.15248/) (2026)

## Abstract (arXiv)

The world consists of objects: distinct entities possessing independent properties and dynamics. For agents to interact with the world intelligently, they must translate sensory inputs into the bound-together features that describe each object. These object-based representations form a natural basis for planning behavior. Active inference (AIF) is an influential unifying account of perception and action, but existing AIF models have not leveraged this important inductive bias. To remedy this, we introduce 'object-based active inference' (OBAI), marrying AIF with recent deep object-based neural networks. OBAI represents distinct objects with separate variational beliefs, and uses selective attention to route inputs to their corresponding object slots. Object representations are endowed with independent action-based dynamics. The dynamics and generative model are learned from experience with a simple environment (active multi-dSprites). We show that OBAI learns to correctly segment the action-perturbed objects from video input, and to manipulate these objects towards arbitrary goals.