# Observation-Augmented Contextual Multi-Armed Bandits for Robotic Search and Exploration

arXiv: https://arxiv.org/abs/2312.12583 · HTML: https://mlsanity.com/paper/2312.12583/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 32 references that are in the map.
- First posted on arXiv 19 Dec 2023.

## Builds on (most cited first, 7 of 7)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Active Inference in Robotics and Artificial Agents: Survey and Challenges](https://mlsanity.com/paper/2112.01871/) (2021)
- [Partially Observable Markov Decision Processes (POMDPs) and Robotics](https://mlsanity.com/paper/2107.07599/) (2021)
- [An empirical evaluation of active inference in multi-armed bandits](https://mlsanity.com/paper/2101.08699/) (2021)
- [Active Inference for Autonomous Decision-Making with Contextual Multi-Armed Bandits](https://mlsanity.com/paper/2209.09185/) (2022)
- [Probabilistic Semantic Data Association for Collaborative Human-Robot Sensing](https://mlsanity.com/paper/2110.09621/) (2021)
- [Data-Free/Data-Sparse Softmax Parameter Estimation with Structured Class Geometries](https://mlsanity.com/paper/1806.00728/) (2018)

## Built on by (most cited first, 3 of 3)

- [Combinatorial Rising Bandits](https://mlsanity.com/paper/2412.00798/) (2024)
- [Online Pareto-Optimal Decision-Making for Complex Tasks using Active Inference](https://mlsanity.com/paper/2406.11984/) (2024)
- [Mixing Makes Markovian Contexts Cheap for Linear Bandits](https://mlsanity.com/paper/2603.12530/) (2026)

## Abstract (arXiv)

We introduce a new variant of contextual multi-armed bandits (CMABs) called observation-augmented CMABs (OA-CMABs) wherein a robot uses extra outcome observations from an external information source, e.g. humans. In OA-CMABs, external observations are a function of context features and thus provide evidence on top of observed option outcomes to infer hidden parameters. However, if external data is error-prone, measures must be taken to preserve the correctness of inference. To this end, we derive a robust Bayesian inference process for OA-CMABs based on recently developed probabilistic semantic data association techniques, which handle complex mixture model parameter priors and hybrid discrete-continuous observation likelihoods for semantic external data sources. To cope with combined uncertainties in OA-CMABs, we also derive a new active inference algorithm for optimal option selection based on approximate expected free energy minimization. This generalizes prior work on CMAB active inference by accounting for faulty observations and non-Gaussian distributions. Results for a simulated deep space search site selection problem show that, even if incorrect semantic observations are provided externally, e.g. by scientists, efficient decision-making and robust parameter inference are still achieved in a wide variety of conditions.