# BEACON: Belief-Enabled Adaptive CONtrol for Imitation Learning under Uncertainty

arXiv: https://arxiv.org/abs/2609.22730 · HTML: https://mlsanity.com/paper/2609.22730/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 14 of its 15 references that are in the map.
- First posted on arXiv 19 Sep 2026.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 14 of 14)

- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Scaling Test-Time Compute Without Verification or RL is Suboptimal](https://mlsanity.com/paper/2502.12118/) (2025)
- [DensePhysNet: Learning Dense Physical Object Representations via Multi-step Dynamic Interactions](https://mlsanity.com/paper/1906.03853/) (2019)
- [Multi-Stage Cable Routing through Hierarchical Imitation Learning](https://mlsanity.com/paper/2307.08927/) (2023)
- [Learning Belief Representations for Imitation Learning in POMDPs](https://mlsanity.com/paper/1906.09510/) (2019)
- [TuneNet: One-Shot Residual Tuning for System Identification and Sim-to-Real Robot Task Transfer](https://mlsanity.com/paper/1907.11200/) (2019)
- [AdaManip: Adaptive Articulated Object Manipulation Environments and Policy Learning](https://mlsanity.com/paper/2502.11124/) (2025)
- [MTIL: Encoding Full History with Mamba for Temporal Imitation Learning](https://mlsanity.com/paper/2505.12410/) (2025)
- [Tactile Pose Estimation and Policy Learning for Unknown Object Manipulation](https://mlsanity.com/paper/2203.10685/) (2022)
- [Offline RL with Observation Histories: Analyzing and Improving Sample Complexity](https://mlsanity.com/paper/2310.20663/) (2023)
- [Adaptive Articulated Object Manipulation On The Fly with Foundation Model Reasoning and Part Grounding](https://mlsanity.com/paper/2507.18276/) (2025)
- [Towards Autonomous Crop Monitoring: Inserting Sensors in Cluttered Environments](https://mlsanity.com/paper/2311.03697/) (2023)
- [SAVOR: Skill Affordance Learning from Visuo-Haptic Perception for Robot-Assisted Bite Acquisition](https://mlsanity.com/paper/2506.02353/) (2025)
- [Learn from What We HAVE: History-Aware VErifier that Reasons about Past Interactions Online](https://mlsanity.com/paper/2509.00271/) (2025)

## Abstract (arXiv)

Robot manipulation tasks often involve hidden state information that cannot be directly observed and must be inferred through sequential physical interactions. In such partially observable settings, conditioning an imitation learning policy directly on the recent raw observation history leads to poor performance. This is due to state aliasing, wherein identical observations may arise from different hidden states, and the policy receives conflicting action labels for the same input. To enable history-aware disambiguation capability, we propose conditioning a diffusion policy on a structured representation of the hidden states using Bayesian belief that exposes both the current most likely state estimate and the remaining uncertainty. This representation replaces raw history with a structured, compact input, enabling the policy to implicitly modulate between exploratory and exploitative behaviors based on belief uncertainty, without explicit mode switching or reward shaping. We evaluate across two domains with qualitatively different belief representations: a continuous belief for cornstalk gripper alignment via tactile sensing, and a discrete categorical distribution for latched door opening. In both domains, the belief-conditioned policy substantially outperforms the observation-only baseline and approaches privileged ground-truth performance, with ablations illustrating that the policy adapts its exploration behavior depending on the belief uncertainty at inference time.