# Neuroscience-inspired perception-action in robotics: applying active inference for state estimation, control and self-perception

arXiv: https://arxiv.org/abs/2105.04261 · HTML: https://mlsanity.com/paper/2105.04261/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 52 references that are in the map.
- First posted on arXiv 10 May 2021.

## Builds on (most cited first, 15 of 15)

- [Active inference on discrete state-spaces: a synthesis](https://mlsanity.com/paper/2001.07203/) (2020)
- [The free energy principle for action and perception: A mathematical review](https://mlsanity.com/paper/1705.09156/) (2017)
- [Whence the Expected Free Energy?](https://mlsanity.com/paper/2004.08128/) (2020)
- [End-to-End Pixel-Based Deep Active Inference for Body Perception and Action](https://mlsanity.com/paper/2001.05847/) (2019)
- [Automatic structured variational inference](https://mlsanity.com/paper/2002.00643/) (2020)
- [Multimodal VAE Active Inference Controller](https://mlsanity.com/paper/2103.04412/) (2021)
- [Deep Active Inference for Partially Observable MDPs](https://mlsanity.com/paper/2009.03622/) (2020)
- [Adaptive robot body learning and estimation through predictive coding](https://mlsanity.com/paper/1805.03104/) (2018)
- [Predictive Processing in Cognitive Robotics: a Review](https://mlsanity.com/paper/2101.06611/) (2021)
- [A Novel Adaptive Controller for Robot Manipulators based on Active Inference](https://mlsanity.com/paper/1909.12768/) (2019)
- [Robot self/other distinction: active inference meets neural networks learning in a mirror](https://mlsanity.com/paper/2004.05473/) (2020)
- [A deep active inference model of the rubber-hand illusion](https://mlsanity.com/paper/2008.07408/) (2020)
- [Robot in the mirror: toward an embodied computational model of mirror self-recognition](https://mlsanity.com/paper/2011.04485/) (2020)
- [Drifting perceptual patterns suggest prediction errors fusion rather than hypothesis selection: replicating the rubber-hand illusion on a robot](https://mlsanity.com/paper/1806.06809/) (2018)
- [A predictive processing model of perception and action for self-other distinction](https://mlsanity.com/paper/1810.09879/) (2018)

## Built on by (most cited first, 5 of 5)

- [Active Inference in Robotics and Artificial Agents: Survey and Challenges](https://mlsanity.com/paper/2112.01871/) (2021)
- [Adaptation through prediction: multisensory active inference torque control](https://mlsanity.com/paper/2112.06752/) (2021)
- [Deep Active Inference for Pixel-Based Discrete Control: Evaluation on the Car Racing Problem](https://mlsanity.com/paper/2109.04155/) (2021)
- [Object-centric proto-symbolic behavioural reasoning from pixels](https://mlsanity.com/paper/2411.17438/) (2024)
- [Robot Localization and Navigation through Predictive Processing using LiDAR](https://mlsanity.com/paper/2109.04139/) (2021)

## Abstract (arXiv)

Unlike robots, humans learn, adapt and perceive their bodies by interacting with the world. Discovering how the brain represents the body and generates actions is of major importance for robotics and artificial intelligence. Here we discuss how neuroscience findings open up opportunities to improve current estimation and control algorithms in robotics. In particular, how active inference, a mathematical formulation of how the brain resists a natural tendency to disorder, provides a unified recipe to potentially solve some of the major challenges in robotics, such as adaptation, robustness, flexibility, generalization and safe interaction. This paper summarizes some experiments and lessons learned from developing such a computational model on real embodied platforms, i.e., humanoid and industrial robots. Finally, we showcase the limitations and challenges that we are still facing to give robots human-like perception