# Assessing Quality Metrics for Neural Reality Gap Input Mitigation in Autonomous Driving Testing

arXiv: https://arxiv.org/abs/2404.18577 · HTML: https://mlsanity.com/paper/2404.18577/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2024.

## Built on by (most cited first, 15 of 15)

- [The Reality Gap in Robotics: Challenges, Solutions, and Best Practices](https://mlsanity.com/paper/2510.20808/) (2025)
- [Efficient Domain Augmentation for Autonomous Driving Testing Using Diffusion Models](https://mlsanity.com/paper/2409.13661/) (2024)
- [Sim2Real Diffusion: Leveraging Foundation Vision Language Models for Adaptive Automated Driving](https://mlsanity.com/paper/2507.00236/) (2025)
- [Bridging the Gap between Real-world and Synthetic Images for Testing Autonomous Driving Systems](https://mlsanity.com/paper/2408.13950/) (2024)
- [Benchmarking Image Perturbations for Testing Automated Driving Assistance Systems](https://mlsanity.com/paper/2501.12269/) (2025)
- [Targeted Deep Learning System Boundary Testing](https://mlsanity.com/paper/2408.06258/) (2024)
- [When Digital Twins Meet Large Language Models: Realistic, Interactive, and Editable Simulation for Autonomous Driving](https://mlsanity.com/paper/2507.00319/) (2025)
- [Simulator Ensembles for Trustworthy Autonomous Driving Systems Testing](https://mlsanity.com/paper/2503.08936/) (2025)
- [A Multi-Modality Evaluation of the Reality Gap in Autonomous Driving Systems](https://mlsanity.com/paper/2509.22379/) (2025)
- [Effort-Optimized, Accuracy-Driven Labelling and Validation of Test Inputs for DL Systems: A Mixed-Integer Linear Programming Approach](https://mlsanity.com/paper/2507.04990/) (2025)
- [HyperNet-Adaptation for Diffusion-Based Test Case Generation](https://mlsanity.com/paper/2601.15041/) (2026)
- [Latent Regularization in Generative Test Input Generation](https://mlsanity.com/paper/2602.15552/) (2026)
- [From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Driving](https://mlsanity.com/paper/2603.17714/) (2026)
- [Search-based Testing of Vision Language Models for In-Car Scene Understanding](https://mlsanity.com/paper/2607.02300/) (2026)
- [Real-World Perturbation Testing of Autonomous Driving Systems](https://mlsanity.com/paper/2607.04953/) (2026)

## Abstract (arXiv)

Simulation-based testing of automated driving systems (ADS) is the industry standard, being a controlled, safe, and cost-effective alternative to real-world testing. Despite these advantages, virtual simulations often fail to accurately replicate real-world conditions like image fidelity, texture representation, and environmental accuracy. This can lead to significant differences in ADS behavior between simulated and real-world domains, a phenomenon known as the sim2real gap. Researchers have used Image-to-Image (I2I) neural translation to mitigate the sim2real gap, enhancing the realism of simulated environments by transforming synthetic data into more authentic representations of real-world conditions. However, while promising, these techniques may potentially introduce artifacts, distortions, or inconsistencies in the generated data that can affect the effectiveness of ADS testing. In our empirical study, we investigated how the quality of image-to-image (I2I) techniques influences the mitigation of the sim2real gap, using a set of established metrics from the literature. We evaluated two popular generative I2I architectures, pix2pix, and CycleGAN, across two ADS perception tasks at a model level, namely vehicle detection and end-to-end lane keeping, using paired simulated and real-world datasets. Our findings reveal that the effectiveness of I2I architectures varies across different ADS tasks, and existing evaluation metrics do not consistently align with the ADS behavior. Thus, we conducted task-specific fine-tuning of perception metrics, which yielded a stronger correlation. Our findings indicate that a perception metric that incorporates semantic elements, tailored to each task, can facilitate selecting the most appropriate I2I technique for a reliable assessment of the sim2real gap mitigation.