# Hybrid Belief Reinforcement Learning for Efficient Coordinated Spatial Exploration

arXiv: https://arxiv.org/abs/2603.03595 · HTML: https://mlsanity.com/paper/2603.03595/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 41 references that are in the map.
- First posted on arXiv 4 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Transfer Learning in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2009.07888/) (2020)
- [Joint Trajectory and Communication Design for Multi-UAV Enabled Wireless Networks](https://mlsanity.com/paper/1705.02723/) (2017)
- [Deep Reinforcement Learning Based Dynamic Trajectory Control for UAV-assisted Mobile Edge Computing](https://mlsanity.com/paper/1911.03887/) (2019)
- [Spatial and Spatio-Temporal Log-Gaussian Cox Processes: Extending the Geostatistical Paradigm](https://mlsanity.com/paper/1312.6536/) (2013)
- [Evolutionary Multi-Objective Reinforcement Learning Based Trajectory Control and Task Offloading in UAV-Assisted Mobile Edge Computing](https://mlsanity.com/paper/2202.12028/) (2022)
- [Learning-based Methods for Adaptive Informative Path Planning](https://mlsanity.com/paper/2404.06940/) (2024)
- [Adaptive Robotic Information Gathering via Non-Stationary Gaussian Processes](https://mlsanity.com/paper/2306.01263/) (2023)
- [Multi-Agent Reinforcement Learning with Action Masking for UAV-enabled Mobile Communications](https://mlsanity.com/paper/2303.16737/) (2023)

## Abstract (arXiv)

Coordinating multiple autonomous agents to explore and serve spatially heterogeneous demand requires jointly learning unknown spatial patterns and planning trajectories that maximize task performance. Pure model-based approaches provide structured uncertainty estimates but lack adaptive policy learning, while deep reinforcement learning often suffers from poor sample efficiency when spatial priors are absent. This paper presents a hybrid belief-reinforcement learning (HBRL) framework to address this gap. In the first phase, agents construct spatial beliefs using a Log-Gaussian Cox Process (LGCP) and execute information-driven trajectories guided by a Pathwise Mutual Information (PathMI) planner with multi-step lookahead. In the second phase, trajectory control is transferred to a Soft Actor-Critic (SAC) agent, warm-started through dual-channel knowledge transfer: belief state initialization supplies spatial uncertainty, and replay buffer seeding provides demonstration trajectories generated during LGCP exploration. A variance-normalized overlap penalty enables coordinated coverage through shared belief state, permitting cooperative sensing in high-uncertainty regions while discouraging redundant coverage in well-explored areas. The framework is evaluated on a multi-UAV wireless service provisioning task. Results show 10.8% higher cumulative reward and 38% faster convergence over baselines, with ablation studies confirming that dual-channel transfer outperforms either channel alone.