# Vision-Based Runtime Monitoring under Varying Specifications using Semantic Latent Representations

arXiv: https://arxiv.org/abs/2605.13923 · HTML: https://mlsanity.com/paper/2605.13923/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 27 references that are in the map.
- First posted on arXiv 13 May 2026.

## Builds on (most cited first, 19 of 19)

- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [Adaptive Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2106.00170/) (2021)
- [Safe Planning in Dynamic Environments using Conformal Prediction](https://mlsanity.com/paper/2210.10254/) (2022)
- [Adaptive Conformal Prediction for Motion Planning among Dynamic Agents](https://mlsanity.com/paper/2212.00278/) (2022)
- [Robust Online Monitoring of Signal Temporal Logic](https://mlsanity.com/paper/1506.08234/) (2015)
- [Conformal Prediction for STL Runtime Verification](https://mlsanity.com/paper/2211.01539/) (2022)
- [WOMD-LiDAR: Raw Sensor Dataset Benchmark for Motion Forecasting](https://mlsanity.com/paper/2304.03834/) (2023)
- [Mining Parametric Temporal Logic Properties in Model Based Design for Cyber-Physical Systems](https://mlsanity.com/paper/1512.07956/) (2015)
- [On-Line Monitoring for Temporal Logic Robustness](https://mlsanity.com/paper/1408.0045/) (2014)
- [Robust Conformal Prediction for STL Runtime Verification under Distribution Shift](https://mlsanity.com/paper/2311.09482/) (2023)
- [Conformal Quantitative Predictive Monitoring of STL Requirements for Stochastic Processes](https://mlsanity.com/paper/2211.02375/) (2022)
- [PerceMon: Online Monitoring for Perception Systems](https://mlsanity.com/paper/2108.08289/) (2021)
- [Neural Predictive Monitoring under Partial Observability](https://mlsanity.com/paper/2108.07134/) (2021)
- [Formalizing and Evaluating Requirements of Perception Systems for Automated Vehicles using Spatio-Temporal Perception Logic](https://mlsanity.com/paper/2206.14372/) (2022)
- [RTAMT -- Runtime Robustness Monitors with Application to CPS and Robotics](https://mlsanity.com/paper/2501.18608/) (2025)
- [Pretrained Embeddings as a Behavior Specification Mechanism](https://mlsanity.com/paper/2503.02012/) (2025)
- [V-CEM: Bridging Performance and Intervenability in Concept-based Models](https://mlsanity.com/paper/2504.03978/) (2025)
- [Interval Signal Temporal Logic from Natural Inclusion Functions](https://mlsanity.com/paper/2309.10686/) (2023)
- [CBFKIT: A Control Barrier Function Toolbox for Robotics Applications](https://mlsanity.com/paper/2404.07158/) (2024)

## Abstract (arXiv)

We study certified runtime monitoring of past-time signal temporal logic (ptSTL) from visual observations under partial observability. The monitor must infer safety-relevant quantities from images and provide finite-sample guarantees, while being reusable: once trained and calibrated, it should certify any formula in a target fragment without per-formula retraining. For fragments induced by a finite dictionary of temporal atoms, we prove that the semantic basis, the vector of atom robustness scores, is the minimum prediction target within the class of monotone, 1-Lipschitz reusable interfaces: any formula is evaluated by a deterministic decoder derived from the parse tree, and a single conformal calibration pass certifies the entire fragment with no union bound. We also introduce a rolling prediction monitor that predicts only current predicate values and reconstructs temporal history online; this is easier to learn but grows conservative at long horizons. On a pedestrian-crossroad benchmark, rolling achieves tighter certified bounds at short horizons while the semantic-basis monitor is up to 4-times tighter at long horizons. We validate the presented monitors on real-world Waymo driving data, where both monitors satisfy the conformal coverage guarantee empirically.