# Active Inference for Autonomous Decision-Making with Contextual Multi-Armed Bandits

arXiv: https://arxiv.org/abs/2209.09185 · HTML: https://mlsanity.com/paper/2209.09185/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 32 references that are in the map.
- First posted on arXiv 19 Sep 2022.

## Builds on (most cited first, 8 of 8)

- [Algorithms for multi-armed bandit problems](https://mlsanity.com/paper/1402.6028/) (2014)
- [Active Inference in Robotics and Artificial Agents: Survey and Challenges](https://mlsanity.com/paper/2112.01871/) (2021)
- [Partially Observable Markov Decision Processes (POMDPs) and Robotics](https://mlsanity.com/paper/2107.07599/) (2021)
- [The Assistive Multi-Armed Bandit](https://mlsanity.com/paper/1901.08654/) (2019)
- [An empirical evaluation of active inference in multi-armed bandits](https://mlsanity.com/paper/2101.08699/) (2021)
- [Prior Preference Learning from Experts:Designing a Reward with Active Inference](https://mlsanity.com/paper/2101.08937/) (2021)
- [Optimal Continuous State POMDP Planning with Semantic Observations: A Variational Approach](https://mlsanity.com/paper/1807.08229/) (2018)
- [Probabilistic Semantic Data Association for Collaborative Human-Robot Sensing](https://mlsanity.com/paper/2110.09621/) (2021)

## Built on by (most cited first, 6 of 6)

- [Bandit Submodular Maximization for Multi-Robot Coordination in Unpredictable and Partially Observable Environments](https://mlsanity.com/paper/2305.12795/) (2023)
- [Observation-Augmented Contextual Multi-Armed Bandits for Robotic Search and Exploration](https://mlsanity.com/paper/2312.12583/) (2023)
- [Leveraging Untrustworthy Commands for Multi-Robot Coordination in Unpredictable Environments: A Bandit Submodular Maximization Approach](https://mlsanity.com/paper/2309.16161/) (2023)
- [Online Pareto-Optimal Decision-Making for Complex Tasks using Active Inference](https://mlsanity.com/paper/2406.11984/) (2024)
- [Active Inference for an Intelligent Agent in Autonomous Reconnaissance Missions](https://mlsanity.com/paper/2510.17450/) (2025)
- [Expected Free Energy-based Informative Path Planning for Robotic Mars Exploration](https://mlsanity.com/paper/2608.14466/) (2026)

## Abstract (arXiv)

In autonomous robotic decision-making under uncertainty, the tradeoff between exploitation and exploration of available options must be considered. If secondary information associated with options can be utilized, such decision-making problems can often be formulated as contextual multi-armed bandits (CMABs). In this study, we apply active inference, which has been actively studied in the field of neuroscience in recent years, as an alternative action selection strategy for CMABs. Unlike conventional action selection strategies, it is possible to rigorously evaluate the uncertainty of each option when calculating the expected free energy (EFE) associated with the decision agent's probabilistic model, as derived from the free-energy principle. We specifically address the case where a categorical observation likelihood function is used, such that EFE values are analytically intractable. We introduce new approximation methods for computing the EFE based on variational and Laplace approximations. Extensive simulation study results demonstrate that, compared to other strategies, active inference generally requires far fewer iterations to identify optimal options and generally achieves superior cumulative regret, for relatively low extra computational cost.