mlsanityData to 7 Oct 2026

Enhancing Photorealism Enhancement

arXiv 2105.04619 ↗First posted 10 May 2021cs.CV cs.AI cs.GRcited by 74

Not on the map. Why?

Abstract (arXiv)

We present an approach to enhancing the realism of synthetic images. The images are enhanced by a convolutional network that leverages intermediate representations produced by conventional rendering pipelines. The network is trained via a novel adversarial objective, which provides strong supervision at multiple perceptual levels. We analyze scene layout distributions in commonly used datasets and find that they differ in important ways. We hypothesize that this is one of the causes of strong artifacts that can be observed in the results of many prior methods. To address this we propose a new strategy for sampling image patches during training. We also introduce multiple architectural improvements in the deep network modules used for photorealism enhancement. We confirm the benefits of our contributions in controlled experiments and report substantial gains in stability and realism in comparison to recent image-to-image translation methods and a variety of other baselines.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). Builds on 67 of its 106 references that are in the map. First posted on arXiv 10 May 2021.

Builds on

67 of 106 references.

References · 67

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  8. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  9. 2016Improved Techniques for Training GANscited by 2,938
  10. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  11. 2017Billion-scale similarity search with GPUscited by 2,477
  12. 2014Conditional Generative Adversarial Netscited by 2,152
38 more
  1. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  2. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  3. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  4. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  5. 2018Demystifying MMD GANscited by 1,268
  6. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  7. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  8. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  9. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,079
  10. 2016Playing for Data: Ground Truth from Computer Gamescited by 816
  11. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  12. 2018Which Training Methods for GANs do actually Converge?cited by 672
  13. 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 637
  14. 2017Unsupervised Image-to-Image Translation Networkscited by 602
  15. 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 562
  16. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 460
  17. 2017VisDA: The Visual Domain Adaptation Challengecited by 432
  18. 2018Video-to-Video Synthesiscited by 388
  19. 2017Universal Style Transfer via Feature Transformscited by 352
  20. 2017DualGAN: Unsupervised Dual Learning for Image-to-Image Translationcited by 310
  21. 2016Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networkscited by 307
  22. 2018Diverse Image-to-Image Translation via Disentangled Representationscited by 296
  23. 2016Learning from Simulated and Unsupervised Images through Adversarial Trainingcited by 287
  24. 2018cGANs with Projection Discriminatorcited by 253
  25. 2019SEAN: Image Synthesis with Semantic Region-Adaptive Normalizationcited by 253
  26. 2017Photographic Image Synthesis with Cascaded Refinement Networkscited by 223
  27. 2020Swapping Autoencoder for Deep Image Manipulationcited by 215
  28. 2016Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networkscited by 192
  29. 2019U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translationcited by 186
  30. 2017Playing for Benchmarkscited by 169
  31. 2017Deep Photo Style Transfercited by 168
  32. 2019Photorealistic Style Transfer via Wavelet Transformscited by 166
  33. 2020Stable View Synthesiscited by 165
  34. 2018A Closed-form Solution to Photorealistic Image Stylizationcited by 133
  35. 2020You Only Need Adversarial Supervision for Semantic Image Synthesiscited by 129
  36. 2017Visual Attribute Transfer through Deep Image Analogycited by 123
  37. 2018Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mappingcited by 113
  38. 2019Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesiscited by 108

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 14; 2023: 23; 2024: 13; 2025: 6; 2026: 9.

74 papers.

Sort:

By area:

  1. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  2. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  3. 2021Projected GANs Converge Fastercited by 204
  4. 2021Ensembling Off-the-shelf Models for GAN Trainingcited by 84
  5. 2022Back to the Source: Diffusion-Driven Test-Time Adaptationcited by 71
  6. 2022Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptationcited by 67
  7. 2021Multi-Modal 3D Object Detection in Autonomous Driving: a Surveycited by 63
  8. 2022READ: Large-Scale Neural Scene Rendering for Autonomous Drivingcited by 50
  9. 2023STEPS: Joint Self-supervised Nighttime Image Enhancement and Depth Estimationcited by 36
  10. 2024SimGen: Simulator-conditioned Driving Scene Generationcited by 34
  11. 2021Task2Sim : Towards Effective Pre-training and Transfer from Synthetic Datacited by 26
  12. 2022On the Road to Online Adaptation for Semantic Image Segmentationcited by 23
38 more
  1. 2021Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehiclescited by 22
  2. 2023LightSim: Neural Lighting Simulation for Urban Scenescited by 20
  3. 2023Single Image Depth Prediction Made Better: A Multivariate Gaussian Takecited by 14
  4. 2023CAD-Estate: Large-scale CAD Model Annotation in RGB Videoscited by 14
  5. 2023Learnable Differencing Center for Nighttime Depth Perceptioncited by 14
  6. 2022Unpaired Image Translation via Vector Symbolic Architecturescited by 13
  7. 2024GenDeg: Diffusion-based Degradation Synthesis for Generalizable All-In-One Image Restorationcited by 13
  8. 2021Towards Optimal Strategies for Training Self-Driving Perception Models in Simulationcited by 12
  9. 2022ADAS: A Direct Adaptation Strategy for Multi-Target Domain Adaptive Semantic Segmentationcited by 10
  10. 2023Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulationcited by 10
  11. 2022Sim2Real Instance-Level Style Transfer for 6D Pose Estimationcited by 9
  12. 2022ProCST: Boosting Semantic Segmentation Using Progressive Cyclic Style-Transfercited by 9
  13. 2022SimVQA: Exploring Simulated Environments for Visual Question Answeringcited by 8
  14. 2024Latency-aware Road Anomaly Segmentation in Videos: A Photorealistic Dataset and New Metricscited by 8
  15. 2024CARLA2Real: a tool for reducing the sim2real appearance gap in CARLA simulatorcited by 8
  16. 2024SEED4D: A Synthetic Ego--Exo Dynamic 4D Data Generator, Driving Dataset and Benchmarkcited by 8
  17. 2023Visual DNA: Representing and Comparing Images using Distributions of Neuron Activationscited by 7
  18. 2023Estimating Generic 3D Room Structures from 2D Annotationscited by 6
  19. 2023Unlocking Pre-trained Image Backbones for Semantic Image Synthesiscited by 6
  20. 2024ContextualFusion: Context-Based Multi-Sensor Fusion for 3D Object Detection in Adverse Operating Conditionscited by 6
  21. 2022Photo-realistic Neural Domain Randomizationcited by 5
  22. 2023Enhanced Stable View Synthesiscited by 5
  23. 2024Synthetic to Authentic: Transferring Realism to 3D Face Renderings for Boosting Face Recognitioncited by 5
  24. 2026Driving with DINO: Vision Foundation Features as a Unified Bridge for Sim-to-Real Generation in Autonomous Drivingcited by 5
  25. 2026Marionette: Predicting World States, Rendering Geometry, Painting Appearancecited by 5
  26. 2023Towards Pragmatic Semantic Image Synthesis for Urban Scenescited by 4
  27. 2024Vision Technologies with Applications in Traffic Surveillance Systems: A Holistic Surveycited by 4
  28. 2022Neural Neural Textures Make Sim2Real Consistentcited by 3
  29. 2022StyleFlow For Content-Fixed Image to Image Translationcited by 3
  30. 2025REGEN: Real-Time Photorealism Enhancement in Games via a Dual-Stage Generative Network Frameworkcited by 3
  31. 2023Hierarchy Flow For High-Fidelity Image-to-Image Translationcited by 2
  32. 2023Masked Discriminators for Content-Consistent Unpaired Image-to-Image Translationcited by 2
  33. 2023Retargeting Visual Data with Deformation Fieldscited by 2
  34. 2023Neural Style Transfer for Computer Gamescited by 2
  35. 2025Beyond Reconstruction: A Physics Based Neural Deferred Shader for Photo-realistic Renderingcited by 2
  36. 2025A Style-Based Profiling Framework for Quantifying the Synthetic-to-Real Gap in Autonomous Driving Datasetscited by 2
  37. 2021The Large Labelled Logo Dataset (L3D): A Multipurpose and Hand-Labelled Continuously Growing Datasetcited by 1
  38. 2023Unpaired Translation from Semantic Label Maps to Images by Leveraging Domain-Specific Simulationscited by 1

50 most cited of 74 All 74 (JSON)