# REGEN: Real-Time Photorealism Enhancement in Games via a Dual-Stage Generative Network Framework

arXiv: https://arxiv.org/abs/2508.17061 · HTML: https://mlsanity.com/paper/2508.17061/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 40 references that are in the map.
- First posted on arXiv 23 Aug 2025.

## Builds on (most cited first, 26 of 26)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Rethinking FID: Towards a Better Evaluation Metric for Image Generation](https://mlsanity.com/paper/2401.09603/) (2023)
- [MSeg: A Composite Dataset for Multi-domain Semantic Segmentation](https://mlsanity.com/paper/2112.13762/) (2021)
- [Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control](https://mlsanity.com/paper/2503.14492/) (2025)
- [Enhancing Photorealism Enhancement](https://mlsanity.com/paper/2105.04619/) (2021)
- [Dual Contrastive Learning for Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/2104.07689/) (2021)
- [Spatially-Adaptive Pixelwise Networks for Fast Image Translation](https://mlsanity.com/paper/2012.02992/) (2020)
- [Generative Artificial Intelligence: A Systematic Review and Applications](https://mlsanity.com/paper/2405.11029/) (2024)
- [Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehicles](https://mlsanity.com/paper/2112.12252/) (2021)
- [CARLA2Real: a tool for reducing the sim2real appearance gap in CARLA simulator](https://mlsanity.com/paper/2410.18238/) (2024)
- [Optical Flow Dataset and Benchmark for Visual Crowd Analysis](https://mlsanity.com/paper/1811.07170/) (2018)
- [Unpaired Image-to-Image Translation with Content Preserving Perspective: A Review](https://mlsanity.com/paper/2502.08667/) (2025)
- [Semantically Consistent Image-to-Image Translation for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2111.03522/) (2021)
- [From Gaming to Research: GTA V for Synthetic Data Generation for Robotics and Navigations](https://mlsanity.com/paper/2502.12303/) (2025)

## Built on by (most cited first, 3 of 3)

- [HyPER-GAN: Hybrid Patch-Based Image-to-Image Translation for Real-Time Photorealism Enhancement in Game Engines](https://mlsanity.com/paper/2603.10604/) (2026)
- [A Hybrid Approach for Closing the Sim2real Appearance Gap in Game Engine Synthetic Datasets](https://mlsanity.com/paper/2605.02291/) (2026)
- [The Domain Is a Residue: Adapting Self-Supervised Features, Not Generators](https://mlsanity.com/paper/2609.37330/) (2026)

## Abstract (arXiv)

Photorealism is an important aspect of modern video games since it can shape player experience and impact immersion, narrative engagement, and visual fidelity. To achieve photorealism, beyond traditional rendering pipelines, generative models have been increasingly adopted as an effective approach for bridging the gap between the visual realism of synthetic and real worlds. However, under real-time constraints of video games, existing generative approaches continue to face a tradeoff between visual quality and runtime efficiency. In this work, we present a framework for enhancing the photorealism of rendered game frames using generative networks. We propose REGEN, which first employs a robust unpaired image-to-image translation model to generate semantically consistent photorealistic frames. These generated frames are then used to create a paired dataset, which transforms the problem to a simpler unpaired image-to-image translation. This enables training with a lightweight method, achieving real-time inference without compromising visual quality. We evaluate REGEN on Unreal Engine, showing, by employing the CMMD metric, that it achieves comparable or slightly improved visual quality compared to the robust method, while improving the frame rate by 12x. Additional experiments also validate that REGEN adheres to the semantic preservation of the initial robust image-to-image translation method and maintains temporal consistency. Code, pre-trained models, and demos for this work are available at: https://github.com/stefanos50/REGEN