# Deep Evaluation Metric: Learning to Evaluate Simulated Radar Point Clouds for Virtual Testing of Autonomous Driving

arXiv: https://arxiv.org/abs/2104.06772 · HTML: https://mlsanity.com/paper/2104.06772/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 29 references that are in the map.
- First posted on arXiv 14 Apr 2021.

## Builds on (most cited first, 5 of 5)

- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [2D Car Detection in Radar Data with PointNets](https://mlsanity.com/paper/1904.08414/) (2019)
- [Polarimetric Radar Cross-Sections of Pedestrians at Automotive Radar Frequencies](https://mlsanity.com/paper/1910.13706/) (2019)
- [A Sensitivity Analysis Approach for Evaluating a Radar Simulation for Virtual Testing of Autonomous Driving Functions](https://mlsanity.com/paper/2008.02725/) (2020)

## Built on by (most cited first, 3 of 3)

- [A Multi-Layered Approach for Measuring the Simulation-to-Reality Gap of Radar Perception for Autonomous Driving](https://mlsanity.com/paper/2106.08372/) (2021)
- [From Model-Based to Data-Driven Simulation: Challenges and Trends in Autonomous Driving](https://mlsanity.com/paper/2305.13960/) (2023)
- [FAKEPCD: Fake Point Cloud Detection via Source Attribution](https://mlsanity.com/paper/2312.11213/) (2023)

## Abstract (arXiv)

The usage of environment sensor models for virtual testing is a promising approach to reduce the testing effort of autonomous driving. However, in order to deduce any statements regarding the performance of an autonomous driving function based on simulation, the sensor model has to be validated to determine the discrepancy between the synthetic and real sensor data. Since a certain degree of divergence can be assumed to exist, the sufficient level of fidelity must be determined, which poses a major challenge. In particular, a method for quantifying the fidelity of a sensor model does not exist and the problem of defining an appropriate metric remains. In this work, we train a neural network to distinguish real and simulated radar sensor data with the purpose of learning the latent features of real radar point clouds. Furthermore, we propose the classifier's confidence score for the `real radar point cloud' class as a metric to determine the degree of fidelity of synthetically generated radar data. The presented approach is evaluated and it can be demonstrated that the proposed deep evaluation metric outperforms conventional metrics in terms of its capability to identify characteristic differences between real and simulated radar data.