# Spatial Language Likelihood Grounding Network for Bayesian Fusion of Human-Robot Observations

arXiv: https://arxiv.org/abs/2507.19947 · HTML: https://mlsanity.com/paper/2507.19947/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 47 references that are in the map.
- First posted on arXiv 26 Jul 2025.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Few-Shot Adversarial Learning of Realistic Neural Talking Head Models](https://mlsanity.com/paper/1905.08233/) (2019)
- [Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight](https://mlsanity.com/paper/1910.09664/) (2019)
- [Revisiting Large Language Models as Zero-shot Relation Extractors](https://mlsanity.com/paper/2310.05028/) (2023)
- [Deep Probability Estimation](https://mlsanity.com/paper/2111.10734/) (2021)
- [Unifying Foundation Models with Quadrotor Control for Visual Tracking Beyond Object Categories](https://mlsanity.com/paper/2310.04781/) (2023)
- [Spatial Language Understanding for Object Search in Partially Observed City-scale Environments](https://mlsanity.com/paper/2012.02705/) (2020)
- [Pedestrian Motion Model Using Non-Parametric Trajectory Clustering and Discrete Transition Points](https://mlsanity.com/paper/2001.10571/) (2020)
- [Probabilistic Semantic Data Association for Collaborative Human-Robot Sensing](https://mlsanity.com/paper/2110.09621/) (2021)
- [Data-Free/Data-Sparse Softmax Parameter Estimation with Structured Class Geometries](https://mlsanity.com/paper/1806.00728/) (2018)
- [HARPS: An Online POMDP Framework for Human-Assisted Robotic Planning and Sensing](https://mlsanity.com/paper/2110.10324/) (2021)
- [Vision-based Relative Detection and Tracking for Teams of Micro Aerial Vehicles](https://mlsanity.com/paper/2207.08301/) (2022)
- [Split Happens! Imprecise and Negative Information in Gaussian Mixture Random Finite Set Filtering](https://mlsanity.com/paper/2207.11356/) (2022)
- [Design and trajectory tracking control of CuRobot: A Cubic Reversible Robot](https://mlsanity.com/paper/2311.16809/) (2023)

## Abstract (arXiv)

Fusing information from human observations can help robots overcome sensing limitations in collaborative tasks. However, an uncertainty-aware fusion framework requires a grounded likelihood representing the uncertainty of human inputs. This paper presents a Feature Pyramid Likelihood Grounding Network (FP-LGN) that grounds spatial language by learning relevant map image features and their relationships with spatial relation semantics. The model is trained as a probability estimator to capture aleatoric uncertainty in human language using three-stage curriculum learning. Results showed that FP-LGN matched expert-designed rules in mean Negative Log-Likelihood (NLL) and demonstrated greater robustness with lower standard deviation. Collaborative sensing results demonstrated that the grounded likelihood successfully enabled uncertainty-aware fusion of heterogeneous human language observations and robot sensor measurements, achieving significant improvements in human-robot collaborative task performance.