# Contextual Safety Reasoning and Grounding for Open-World Robots

arXiv: https://arxiv.org/abs/2602.19983 · HTML: https://mlsanity.com/paper/2602.19983/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 23 of its 65 references that are in the map.
- First posted on arXiv 23 Feb 2026.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 23 of 23)

- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [Depth Anything 3: Recovering the Visual Space from Any Views](https://mlsanity.com/paper/2511.10647/) (2025)
- [Hamilton-Jacobi Reachability: A Brief Overview and Recent Advances](https://mlsanity.com/paper/1709.07523/) (2017)
- [AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents](https://mlsanity.com/paper/2401.12963/) (2024)
- [Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agents](https://mlsanity.com/paper/2309.09919/) (2023)
- [How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics](https://mlsanity.com/paper/1710.11040/) (2017)
- [NL2TL: Transforming Natural Languages to Temporal Logics using Large Language Models](https://mlsanity.com/paper/2305.07766/) (2023)
- [Semantic Anomaly Detection with Large Language Models](https://mlsanity.com/paper/2305.11307/) (2023)
- [Specific versus General Principles for Constitutional AI](https://mlsanity.com/paper/2310.13798/) (2023)
- [VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Models](https://mlsanity.com/paper/2404.00210/) (2024)
- [Generating Robot Constitutions & Benchmarks for Semantic Safety](https://mlsanity.com/paper/2503.08663/) (2025)
- [Embodied AI with Two Arms: Zero-shot Learning, Safety and Modularity](https://mlsanity.com/paper/2404.03570/) (2024)
- [Reasoning about the Unseen for Efficient Outdoor Object Navigation](https://mlsanity.com/paper/2309.10103/) (2023)
- [Measurement-Robust Control Barrier Functions: Certainty in Safety with Uncertainty in State](https://mlsanity.com/paper/2104.14030/) (2021)
- [SHIELD: Safety on Humanoids via CBFs In Expectation on Learned Dynamics](https://mlsanity.com/paper/2505.11494/) (2025)
- [Robust Control Barrier Functions using Uncertainty Estimation with Application to Mobile Robots](https://mlsanity.com/paper/2401.01881/) (2024)
- [From Words to Safety: Language-Conditioned Safety Filtering for Robot Navigation](https://mlsanity.com/paper/2511.05889/) (2025)
- [Robust Safety under Stochastic Uncertainty with Discrete-Time Control Barrier Functions](https://mlsanity.com/paper/2302.07469/) (2023)
- [Generative Modeling of Residuals for Real-Time Risk-Sensitive Safety with Discrete-Time Control Barrier Functions](https://mlsanity.com/paper/2311.05802/) (2023)
- [Robust Control Barrier Functions with Uncertainty Estimation](https://mlsanity.com/paper/2304.08538/) (2023)
- [Safety Under State Uncertainty: Robustifying Control Barrier Functions](https://mlsanity.com/paper/2508.17226/) (2025)
- [Efficient Navigation in Unknown Indoor Environments with Vision-Language Models](https://mlsanity.com/paper/2510.04991/) (2025)

## Built on by (most cited first, 4 of 4)

- [HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Task](https://mlsanity.com/paper/2603.14367/) (2026)
- [Using large language models for embodied planning introduces systematic safety risks](https://mlsanity.com/paper/2604.18463/) (2026)
- [LiSA: Lifelong Safety Adaptation via Conservative Policy Induction](https://mlsanity.com/paper/2605.14454/) (2026)
- [Safe Embodied AI for Long-horizon Tasks: A Cross-layer Analysis of Robotic Manipulation](https://mlsanity.com/paper/2606.05660/) (2026)

## Abstract (arXiv)

Robots are increasingly operating in open-world environments where safe behavior depends on context: the same hallway may require different navigation strategies when crowded versus empty, or during an emergency versus normal operations. Traditional safety approaches enforce fixed constraints in user-specified contexts, limiting their ability to handle the open-ended contextual variability of real-world deployment. We address this gap via CORE, a safety framework that enables online contextual reasoning, grounding, and enforcement without prior knowledge of the environment (e.g., maps or safety specifications). CORE uses a vision-language model (VLM) to continuously reason about context-dependent safety rules directly from visual observations, grounds these rules in the physical environment, and enforces the resulting spatially-defined safe sets via control barrier functions. We provide probabilistic safety guarantees for CORE that account for perceptual uncertainty, and we demonstrate through simulation and real-world experiments that CORE enforces contextually appropriate behavior in unseen environments, significantly outperforming prior semantic safety methods that lack online contextual reasoning. Ablation studies validate our theoretical guarantees and underscore the importance of both VLM-based reasoning and spatial grounding for enforcing contextual safety in novel settings. We provide additional resources at https://zacravichandran.github.io/CORE.