# Adaptation through prediction: multisensory active inference torque control

arXiv: https://arxiv.org/abs/2112.06752 · HTML: https://mlsanity.com/paper/2112.06752/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 13 Dec 2021.

## Builds on (most cited first, 14 of 14)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [State Representation Learning for Control: An Overview](https://mlsanity.com/paper/1802.04181/) (2018)
- [The free energy principle for action and perception: A mathematical review](https://mlsanity.com/paper/1705.09156/) (2017)
- [On the Relationship Between Active Inference and Control as Inference](https://mlsanity.com/paper/2006.12964/) (2020)
- [End-to-End Pixel-Based Deep Active Inference for Body Perception and Action](https://mlsanity.com/paper/2001.05847/) (2019)
- [Multimodal VAE Active Inference Controller](https://mlsanity.com/paper/2103.04412/) (2021)
- [Adaptive robot body learning and estimation through predictive coding](https://mlsanity.com/paper/1805.03104/) (2018)
- [Predictive Processing in Cognitive Robotics: a Review](https://mlsanity.com/paper/2101.06611/) (2021)
- [A Novel Adaptive Controller for Robot Manipulators based on Active Inference](https://mlsanity.com/paper/1909.12768/) (2019)
- [Robot self/other distinction: active inference meets neural networks learning in a mirror](https://mlsanity.com/paper/2004.05473/) (2020)
- [Active Inference for Integrated State-Estimation, Control, and Learning](https://mlsanity.com/paper/2005.05894/) (2020)
- [Goal-Directed Behavior under Variational Predictive Coding: Dynamic Organization of Visual Attention and Working Memory](https://mlsanity.com/paper/1903.04932/) (2019)
- [Fault-tolerant Control of Robot Manipulators with Sensory Faults using Unbiased Active Inference](https://mlsanity.com/paper/2104.01817/) (2021)
- [Neuroscience-inspired perception-action in robotics: applying active inference for state estimation, control and self-perception](https://mlsanity.com/paper/2105.04261/) (2021)

## Built on by (most cited first, 6 of 6)

- [Goal-directed Planning and Goal Understanding by Active Inference: Evaluation Through Simulated and Physical Robot Experiments](https://mlsanity.com/paper/2202.09976/) (2022)
- [Learning Policies for Continuous Control via Transition Models](https://mlsanity.com/paper/2209.08033/) (2022)
- [TC-VAE: Uncovering Out-of-Distribution Data Generative Factors](https://mlsanity.com/paper/2304.04103/) (2023)
- [Dynamic planning in hierarchical active inference](https://mlsanity.com/paper/2402.11658/) (2024)
- [Seeing Through Uncertainty: Free-Energy-Inspired Real-Time Adaptation for Robust Visual Navigation](https://mlsanity.com/paper/2403.01977/) (2024)
- [Active Inference for an Intelligent Agent in Autonomous Reconnaissance Missions](https://mlsanity.com/paper/2510.17450/) (2025)

## Abstract (arXiv)

Adaptation to external and internal changes is major for robotic systems in uncertain environments. Here we present a novel multisensory active inference torque controller for industrial arms that shows how prediction can be used to resolve adaptation. Our controller, inspired by the predictive brain hypothesis, improves the capabilities of current active inference approaches by incorporating learning and multimodal integration of low and high-dimensional sensor inputs (e.g., raw images) while simplifying the architecture. We performed a systematic evaluation of our model on a 7DoF Franka Emika Panda robot arm by comparing its behavior with previous active inference baselines and classic controllers, analyzing both qualitatively and quantitatively adaptation capabilities and control accuracy. Results showed improved control accuracy in goal-directed reaching with high noise rejection due to multimodal filtering, and adaptability to dynamical inertial changes, elasticity constraints and human disturbances without the need to relearn the model nor parameter retuning.