# HARPS: An Online POMDP Framework for Human-Assisted Robotic Planning and Sensing

arXiv: https://arxiv.org/abs/2110.10324 · HTML: https://mlsanity.com/paper/2110.10324/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 78 references that are in the map.
- First posted on arXiv 20 Oct 2021.

## Builds on (most cited first, 9 of 9)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [DESPOT: Online POMDP Planning with Regularization](https://mlsanity.com/paper/1609.03250/) (2016)
- [Online algorithms for POMDPs with continuous state, action, and observation spaces](https://mlsanity.com/paper/1709.06196/) (2017)
- [Optimal Continuous State POMDP Planning with Semantic Observations: A Variational Approach](https://mlsanity.com/paper/1807.08229/) (2018)
- [Planning with Verbal Communication for Human-Robot Collaboration](https://mlsanity.com/paper/1706.04694/) (2017)
- [Data-Free/Data-Sparse Softmax Parameter Estimation with Structured Class Geometries](https://mlsanity.com/paper/1806.00728/) (2018)
- [Deep Value of Information Estimators for Collaborative Human-Machine Information Gathering](https://mlsanity.com/paper/1512.07592/) (2015)
- [Closed-loop Bayesian Semantic Data Fusion for Collaborative Human-Autonomy Target Search](https://mlsanity.com/paper/1806.00727/) (2018)

## Built on by (most cited first, 2 of 2)

- [Spatial Language Likelihood Grounding Network for Bayesian Fusion of Human-Robot Observations](https://mlsanity.com/paper/2507.19947/) (2025)
- [C-IDS: Solving Contextual POMDP via Information-Directed Objective](https://mlsanity.com/paper/2602.03939/) (2026)

## Abstract (arXiv)

Autonomous robots can benefit greatly from human-provided semantic characterizations of uncertain task environments and states. However, the development of integrated strategies which let robots model, communicate, and act on such 'soft data' remains challenging. Here, the Human Assisted Robotic Planning and Sensing (HARPS) framework is presented for active semantic sensing and planning in human-robot teams to address these gaps by formally combining the benefits of online sampling-based POMDP policies, multimodal semantic interaction, and Bayesian data fusion. This approach lets humans opportunistically impose model structure and extend the range of semantic soft data in uncertain environments by sketching and labeling arbitrary landmarks across the environment. Dynamic updating of the environment model while during search allows robotic agents to actively query humans for novel and relevant semantic data, thereby improving beliefs of unknown environments and states for improved online planning. Simulations of a UAV-enabled target search application in a large-scale partially structured environment show significant improvements in time and belief state estimates required for interception versus conventional planning based solely on robotic sensing. Human subject studies in the same environment (n = 36) demonstrate an average doubling in dynamic target capture rate compared to the lone robot case, and highlight the robustness of active probabilistic reasoning and semantic sensing over a range of user characteristics and interaction modalities.