# PEM: Perception Error Model for Virtual Testing of Autonomous Vehicles

arXiv: https://arxiv.org/abs/2302.11919 · HTML: https://mlsanity.com/paper/2302.11919/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 52 references that are in the map.
- First posted on arXiv 23 Feb 2023.

## Builds on (most cited first, 14 of 14)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [On a Formal Model of Safe and Scalable Self-driving Cars](https://mlsanity.com/paper/1708.06374/) (2017)
- [LGSVL Simulator: A High Fidelity Simulator for Autonomous Driving](https://mlsanity.com/paper/2005.03778/) (2020)
- [Baidu Apollo EM Motion Planner](https://mlsanity.com/paper/1807.08048/) (2018)
- [Testing Scenario Library Generation for Connected and Automated Vehicles, Part I: Methodology](https://mlsanity.com/paper/1905.03419/) (2019)
- [To Explain or Not to Explain: A Study on the Necessity of Explanations for Autonomous Vehicles](https://mlsanity.com/paper/2006.11684/) (2020)
- [Towards Dependability Metrics for Neural Networks](https://mlsanity.com/paper/1806.02338/) (2018)
- [A Review of Testing Object-Based Environment Perception for Safe Automated Driving](https://mlsanity.com/paper/2102.08460/) (2021)
- [Testing Rare Downstream Safety Violations via Upstream Adaptive Sampling of Perception Error Models](https://mlsanity.com/paper/2209.09674/) (2022)
- [ViSTA: a Framework for Virtual Scenario-based Testing of Autonomous Vehicles](https://mlsanity.com/paper/2109.02529/) (2021)
- [A Step Towards Efficient Evaluation of Complex Perception Tasks in Simulation](https://mlsanity.com/paper/2110.02739/) (2021)
- [Safety Metrics for Semantic Segmentation in Autonomous Driving](https://mlsanity.com/paper/2105.10142/) (2021)

## Built on by (most cited first, 4 of 4)

- [Adaptive Splitting of Reusable Temporal Monitors for Rare Traffic Violations](https://mlsanity.com/paper/2405.15771/) (2024)
- [Control Analysis and Design for Autonomous Vehicles Subject to Imperfect AI-Based Perception](https://mlsanity.com/paper/2509.12137/) (2025)
- [Approaches to Analysis and Design of AI-Based Autonomous Vehicles](https://mlsanity.com/paper/2509.12169/) (2025)
- [Online Monitoring Framework for Automotive Time Series Data using JEPA Embeddings](https://mlsanity.com/paper/2602.09985/) (2026)

## Abstract (arXiv)

Even though virtual testing of Autonomous Vehicles (AVs) has been well recognized as essential for safety assessment, AV simulators are still undergoing active development. One particularly challenging question is to effectively include the Sensing and Perception (S&P) subsystem into the simulation loop. In this article, we define Perception Error Models (PEM), a virtual simulation component that can enable the analysis of the impact of perception errors on AV safety, without the need to model the sensors themselves. We propose a generalized data-driven procedure towards parametric modeling and evaluate it using Apollo, an open-source driving software, and nuScenes, a public AV dataset. Additionally, we implement PEMs in SVL, an open-source vehicle simulator. Furthermore, we demonstrate the usefulness of PEM-based virtual tests, by evaluating camera, LiDAR, and camera-LiDAR setups. Our virtual tests highlight limitations in the current evaluation metrics, and the proposed approach can help study the impact of perception errors on AV safety.