# End-to-End Pixel-Based Deep Active Inference for Body Perception and Action

arXiv: https://arxiv.org/abs/2001.05847 · HTML: https://mlsanity.com/paper/2001.05847/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Dec 2019.

## Built on by (most cited first, 24 of 24)

- [Active Inference in Robotics and Artificial Agents: Survey and Challenges](https://mlsanity.com/paper/2112.01871/) (2021)
- [Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?](https://mlsanity.com/paper/2202.09467/) (2022)
- [World Models and Predictive Coding for Cognitive and Developmental Robotics: Frontiers and Challenges](https://mlsanity.com/paper/2301.05832/) (2023)
- [The Free Energy Principle for Perception and Action: A Deep Learning Perspective](https://mlsanity.com/paper/2207.06415/) (2022)
- [Multimodal VAE Active Inference Controller](https://mlsanity.com/paper/2103.04412/) (2021)
- [Branching Time Active Inference: the theory and its generality](https://mlsanity.com/paper/2111.11107/) (2021)
- [Branching Time Active Inference: empirical study and complexity class analysis](https://mlsanity.com/paper/2111.11276/) (2021)
- [Adaptation through prediction: multisensory active inference torque control](https://mlsanity.com/paper/2112.06752/) (2021)
- [Disentangling Shape and Pose for Object-Centric Deep Active Inference Models](https://mlsanity.com/paper/2209.09097/) (2022)
- [Neuroscience-inspired perception-action in robotics: applying active inference for state estimation, control and self-perception](https://mlsanity.com/paper/2105.04261/) (2021)
- [Deconstructing deep active inference](https://mlsanity.com/paper/2303.01618/) (2023)
- [Learning Policies for Continuous Control via Transition Models](https://mlsanity.com/paper/2209.08033/) (2022)
- [Deep Active Inference for Pixel-Based Discrete Control: Evaluation on the Car Racing Problem](https://mlsanity.com/paper/2109.04155/) (2021)
- [Branching Time Active Inference with Bayesian Filtering](https://mlsanity.com/paper/2112.07406/) (2021)
- [Multi-Modal and Multi-Factor Branching Time Active Inference](https://mlsanity.com/paper/2206.12503/) (2022)
- [Free Energy Projective Simulation (FEPS): Active inference with interpretability](https://mlsanity.com/paper/2411.14991/) (2024)
- [Robot Localization and Navigation through Predictive Processing using LiDAR](https://mlsanity.com/paper/2109.04139/) (2021)
- [Emergence of Goal-Directed Behaviors via Active Inference with Self-Prior](https://mlsanity.com/paper/2504.11075/) (2025)
- [Visual Sensation and Perception Computational Models for Deep Learning: State of the art, Challenges and Prospects](https://mlsanity.com/paper/2109.03391/) (2021)
- [Integrating large language models and active inference to understand eye movements in reading and dyslexia](https://mlsanity.com/paper/2308.04941/) (2023)
- [Deep hybrid models: infer and plan in a dynamic world](https://mlsanity.com/paper/2402.10088/) (2024)
- [Dynamic planning in hierarchical active inference](https://mlsanity.com/paper/2402.11658/) (2024)
- [Ego-Foresight: Self-supervised Learning of Agent-Aware Representations for Improved RL](https://mlsanity.com/paper/2407.01570/) (2024)
- [A Hardware-oriented Approach for Efficient Bayesian Inference Computation and Deployment](https://mlsanity.com/paper/2607.17855/) (2026)

## Abstract (arXiv)

We present a pixel-based deep active inference algorithm (PixelAI) inspired by human body perception and action. Our algorithm combines the free-energy principle from neuroscience, rooted in variational inference, with deep convolutional decoders to scale the algorithm to directly deal with raw visual input and provide online adaptive inference. Our approach is validated by studying body perception and action in a simulated and a real Nao robot. Results show that our approach allows the robot to perform 1) dynamical body estimation of its arm using only monocular camera images and 2) autonomous reaching to "imagined" arm poses in the visual space. This suggests that robot and human body perception and action can be efficiently solved by viewing both as an active inference problem guided by ongoing sensory input.