# Predictive Semantic Safety: From Visual Physical Reasoning to Safety-Critical Control

arXiv: https://arxiv.org/abs/2609.34356 · HTML: https://mlsanity.com/paper/2609.34356/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 20 of its 23 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 20 of 20)

- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Control Barrier Function Based Quadratic Programs for Safety Critical Systems](https://mlsanity.com/paper/1609.06408/) (2016)
- [PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding](https://mlsanity.com/paper/2501.16411/) (2025)
- [Physion: Evaluating Physical Prediction from Vision in Humans and Machines](https://mlsanity.com/paper/2106.08261/) (2021)
- [Safe Planning in Dynamic Environments using Conformal Prediction](https://mlsanity.com/paper/2210.10254/) (2022)
- [Adaptive Conformal Prediction for Motion Planning among Dynamic Agents](https://mlsanity.com/paper/2212.00278/) (2022)
- [Learning Physical Dynamics with Subequivariant Graph Neural Networks](https://mlsanity.com/paper/2210.06876/) (2022)
- [OmniVLA: An Omni-Modal Vision-Language-Action Model for Robot Navigation](https://mlsanity.com/paper/2509.19480/) (2025)
- [Control Barrier Functions for Systems with High Relative Degree](https://mlsanity.com/paper/1903.04706/) (2019)
- [Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning](https://mlsanity.com/paper/2608.27549/) (2026)
- [Backup Control Barrier Functions: Formulation and Comparative Study](https://mlsanity.com/paper/2104.11332/) (2021)
- [Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoning](https://mlsanity.com/paper/2505.10547/) (2025)
- [Signal Temporal Logic Control Synthesis among Uncontrollable Dynamic Agents with Conformal Prediction](https://mlsanity.com/paper/2312.04242/) (2023)
- [Recursively Feasible Shrinking-Horizon MPC in Dynamic Environments with Conformal Prediction Guarantees](https://mlsanity.com/paper/2405.10875/) (2024)
- [Learning to Adapt Control Barrier Functions Under Epistemic and Aleatoric Uncertainty](https://mlsanity.com/paper/2504.03038/) (2025)
- [Is Your Safe Controller Actually Safe? A Critical Review of CBF Tautologies and Hidden Assumptions](https://mlsanity.com/paper/2603.06954/) (2026)
- [Backup-Based Safety Filters: A Comparative Review of Backup CBF, Model Predictive Shielding, and gatekeeper](https://mlsanity.com/paper/2604.02401/) (2026)
- [Policy Library CBF: Finite-Horizon Safety at Runtime via Parallel Rollouts](https://mlsanity.com/paper/2605.16588/) (2026)
- [OcclusionCBF: Backup Control Barrier Functions for Safe Navigation Among Hidden Dynamic Obstacles](https://mlsanity.com/paper/2609.06342/) (2026)

## Abstract (arXiv)

Physical interactions can create future hazards that are not apparent from the robot's current geometric surroundings. We present a framework termed Predictive Semantic Safety (PSS), which connects visual physical reasoning to backup-based safety filtering. A vision-language model (VLM) predicts physical events and their timing or directly predicts object displacements. An explicit motion model converts event hypotheses into object trajectories. Split conformal prediction calibrates position errors jointly across specified objects, observation times, and future times; geometric shape bounds convert the resulting position regions into predicted object occupancy. PSS evaluates a prescribed backup maneuver against this occupancy and derives input-affine constraints for minimally modifying the nominal input while preserving backup feasibility under the robot dynamics and input limits. MuJoCo experiments with a Unitree Go1 consider falling fixtures, impact-driven support loss, and contact propagation. PSS achieves a safe episode rate of 99.3%, compared with 43.3% for a Backup Control Barrier Function baseline that only uses current obstacle geometry.