# Formalizing and Evaluating Requirements of Perception Systems for Automated Vehicles using Spatio-Temporal Perception Logic

arXiv: https://arxiv.org/abs/2206.14372 · HTML: https://mlsanity.com/paper/2206.14372/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 94 references that are in the map.
- First posted on arXiv 29 Jun 2022.

## Builds on (most cited first, 21 of 21)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [A Survey of Autonomous Driving: Common Practices and Emerging Technologies](https://mlsanity.com/paper/1906.05113/) (2019)
- [Playing for Benchmarks](https://mlsanity.com/paper/1709.07322/) (2017)
- [A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems](https://mlsanity.com/paper/2005.02979/) (2020)
- [SqueezeDet: Unified, Small, Low Power Fully Convolutional Neural Networks for Real-Time Object Detection for Autonomous Driving](https://mlsanity.com/paper/1612.01051/) (2016)
- [Formal Scenario-Based Testing of Autonomous Vehicles: From Simulation to the Real World](https://mlsanity.com/paper/2003.07739/) (2020)
- [Compositional Falsification of Cyber-Physical Systems with Machine Learning Components](https://mlsanity.com/paper/1703.00978/) (2017)
- [Requirements-driven Test Generation for Autonomous Vehicles with Machine Learning Components](https://mlsanity.com/paper/1908.01094/) (2019)
- [PerceMon: Online Monitoring for Perception Systems](https://mlsanity.com/paper/2108.08289/) (2021)
- [Monitoring and Diagnosability of Perception Systems](https://mlsanity.com/paper/2011.07010/) (2020)
- [DeepCrashTest: Turning Dashcam Videos into Virtual Crash Tests for Automated Driving Systems](https://mlsanity.com/paper/2003.11766/) (2020)
- [Predictive Monitoring with Logic-Calibrated Uncertainty for Cyber-Physical Systems](https://mlsanity.com/paper/2011.00384/) (2020)
- [VidCEP: Complex Event Processing Framework to Detect Spatiotemporal Patterns in Video Streams](https://mlsanity.com/paper/2007.07817/) (2020)
- [Spatial Logics and Model Checking for Medical Imaging (Extended Version)](https://mlsanity.com/paper/1811.06065/) (2018)
- [A Formal Methods Approach to Pattern Synthesis in Reaction Diffusion Systems](https://mlsanity.com/paper/1409.5671/) (2014)
- [A Survey of Qualitative Spatial and Temporal Calculi -- Algebraic and Computational Properties](https://mlsanity.com/paper/1606.00133/) (2016)
- [A graph-based spatial temporal logic for knowledge representation and automated reasoning in cognitive robots](https://mlsanity.com/paper/2001.07205/) (2020)
- [Querying Labelled Data with Scenario Programs for Sim-to-Real Validation](https://mlsanity.com/paper/2112.00206/) (2021)
- [An Efficient Algorithm for Monitoring Practical TPTL Specifications](https://mlsanity.com/paper/1612.03140/) (2016)
- [Proof Theory of a Multi-Lane Spatial Logic](https://mlsanity.com/paper/1504.06986/) (2015)

## Built on by (most cited first, 5 of 5)

- [Pretrained Embeddings as a Behavior Specification Mechanism](https://mlsanity.com/paper/2503.02012/) (2025)
- [Runtime Monitoring of Perception-Based Autonomous Systems via Embedding Temporal Logic](https://mlsanity.com/paper/2605.12651/) (2026)
- [Querying Perception Streams with Spatial Regular Expressions](https://mlsanity.com/paper/2411.05946/) (2024)
- [Vision-Based Runtime Monitoring under Varying Specifications using Semantic Latent Representations](https://mlsanity.com/paper/2605.13923/) (2026)
- [Engineering Reliable Autonomous Systems: Challenges and Solutions](https://mlsanity.com/paper/2606.23760/) (2026)

## Abstract (arXiv)

Automated vehicles (AV) heavily depend on robust perception systems. Current methods for evaluating vision systems focus mainly on frame-by-frame performance. Such evaluation methods appear to be inadequate in assessing the performance of a perception subsystem when used within an AV. In this paper, we present a logic -- referred to as Spatio-Temporal Perception Logic (STPL) -- which utilizes both spatial and temporal modalities. STPL enables reasoning over perception data using spatial and temporal operators. One major advantage of STPL is that it facilitates basic sanity checks on the functional performance of the perception system, even without ground-truth data in some cases. We identify a fragment of STPL which is efficiently monitorable offline in polynomial time. Finally, we present a range of specifications for AV perception systems to highlight the types of requirements that can be expressed and analyzed through offline monitoring with STPL.