# Using simulation to quantify the performance of automotive perception systems

arXiv: https://arxiv.org/abs/2303.00983 · HTML: https://mlsanity.com/paper/2303.00983/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 2 Mar 2023.

## Builds on (most cited first, 9 of 9)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [How to Train Neural Networks for Flare Removal](https://mlsanity.com/paper/2011.12485/) (2020)
- [Flare7K: A Phenomenological Nighttime Flare Removal Dataset](https://mlsanity.com/paper/2210.06570/) (2022)
- [See Clearer at Night: Towards Robust Nighttime Semantic Segmentation through Day-Night Image Conversion](https://mlsanity.com/paper/1908.05868/) (2019)
- [Neural Network Generalization: The impact of camera parameters](https://mlsanity.com/paper/1912.03604/) (2019)
- [Soft Prototyping Camera Designs for Car Detection Based on a Convolutional Neural Network](https://mlsanity.com/paper/1910.10916/) (2019)
- [IMMVP: An Efficient Daytime and Nighttime On-Road Object Detector](https://mlsanity.com/paper/1910.06573/) (2019)
- [ISETAuto: Detecting vehicles with depth and radiance information](https://mlsanity.com/paper/2101.01843/) (2021)

## Built on by (most cited first, 2 of 2)

- [TaCOS: Task-Specific Camera Optimization with Simulation](https://mlsanity.com/paper/2404.11031/) (2024)
- [JOCA: Task-Driven Joint Optimisation of Camera Hardware and Adaptive Camera Control Algorithms](https://mlsanity.com/paper/2512.06763/) (2025)

## Abstract (arXiv)

The design and evaluation of complex systems can benefit from a software simulation - sometimes called a digital twin. The simulation can be used to characterize system performance or to test its performance under conditions that are difficult to measure (e.g., nighttime for automotive perception systems). We describe the image system simulation software tools that we use to evaluate the performance of image systems for object (automobile) detection. We describe experiments with 13 different cameras with a variety of optics and pixel sizes. To measure the impact of camera spatial resolution, we designed a collection of driving scenes that had cars at many different distances. We quantified system performance by measuring average precision and we report a trend relating system resolution and object detection performance. We also quantified the large performance degradation under nighttime conditions, compared to daytime, for all cameras and a COCO pre-trained network.