# A Hybrid Approach for Closing the Sim2real Appearance Gap in Game Engine Synthetic Datasets

arXiv: https://arxiv.org/abs/2605.02291 · HTML: https://mlsanity.com/paper/2605.02291/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 15 references that are in the map.
- First posted on arXiv 4 May 2026.

## Builds on (most cited first, 12 of 12)

- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Virtual KITTI 2](https://mlsanity.com/paper/2001.10773/) (2020)
- [Rethinking FID: Towards a Better Evaluation Metric for Image Generation](https://mlsanity.com/paper/2401.09603/) (2023)
- [YOLO26: Key Architectural Enhancements and Performance Benchmarking for Real-Time Object Detection](https://mlsanity.com/paper/2509.25164/) (2025)
- [Enhancing Photorealism Enhancement](https://mlsanity.com/paper/2105.04619/) (2021)
- [Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehicles](https://mlsanity.com/paper/2112.12252/) (2021)
- [Advances in Diffusion Models for Image Data Augmentation: A Review of Methods, Models, Evaluation Metrics and Future Research Directions](https://mlsanity.com/paper/2407.04103/) (2024)
- [CARLA2Real: a tool for reducing the sim2real appearance gap in CARLA simulator](https://mlsanity.com/paper/2410.18238/) (2024)
- [Sim2Real Diffusion: Leveraging Foundation Vision Language Models for Adaptive Automated Driving](https://mlsanity.com/paper/2507.00236/) (2025)
- [REGEN: Real-Time Photorealism Enhancement in Games via a Dual-Stage Generative Network Framework](https://mlsanity.com/paper/2508.17061/) (2025)
- [Zero-shot Synthetic Video Realism Enhancement via Structure-aware Denoising](https://mlsanity.com/paper/2511.14719/) (2025)

## Abstract (arXiv)

Video game engines have been an important source for generating large volumes of visual synthetic datasets for training and evaluating computer vision algorithms that are to be deployed in the real world. While the visual fidelity of modern game engines has been significantly improved with technologies such as ray-tracing, a notable sim2real appearance gap between the synthetic and the real-world images still remains, which limits the utilization of synthetic datasets in real-world applications. In this letter, we investigate the ability of a state-of-the-art image generation and editing diffusion model (FLUX.2-4B Klein) to enhance the photorealism of synthetic datasets and compare its performance against a traditional image-to-image translation model (REGEN). Furthermore, we propose a hybrid approach that combines the strong geometry and material transformations of diffusion-based methods with the distribution-matching capabilities of image-to-image translation techniques. Through experiments, it is demonstrated that REGEN outperforms FLUX.2-4B Klein and that by combining both FLUX.2-4B Klein and REGEN models, better visual realism can be achieved compared to using each model individually, while maintaining semantic consistency. The code is available at: https://github.com/stefanos50/Hybrid-Sim2Real