# Deep CG2Real: Synthetic-to-Real Translation via Image Disentanglement

arXiv: https://arxiv.org/abs/2003.12649 · HTML: https://mlsanity.com/paper/2003.12649/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2020.

## Built on by (most cited first, 15 of 15)

- [GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation](https://mlsanity.com/paper/2112.08867/) (2021)
- [Enhancing Photorealism Enhancement](https://mlsanity.com/paper/2105.04619/) (2021)
- [AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatars](https://mlsanity.com/paper/2210.06465/) (2022)
- [OmniAvatar: Geometry-Guided Controllable 3D Head Synthesis](https://mlsanity.com/paper/2303.15539/) (2023)
- [Sim2Real Instance-Level Style Transfer for 6D Pose Estimation](https://mlsanity.com/paper/2203.02069/) (2022)
- [Self-Supervised Learning of Domain Invariant Features for Depth Estimation](https://mlsanity.com/paper/2106.02594/) (2021)
- [Physics-informed Guided Disentanglement in Generative Networks](https://mlsanity.com/paper/2107.14229/) (2021)
- [Neural Rendering and Its Hardware Acceleration: A Review](https://mlsanity.com/paper/2402.00028/) (2024)
- [Separating Content and Style for Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/2110.14404/) (2021)
- [FashionR2R: Texture-preserving Rendered-to-Real Image Translation with Diffusion Models](https://mlsanity.com/paper/2410.14429/) (2024)
- [Leveraging in-domain supervision for unsupervised image-to-image translation tasks via multi-stream generators](https://mlsanity.com/paper/2112.15091/) (2021)
- [RiCS: A 2D Self-Occlusion Map for Harmonizing Volumetric Objects](https://mlsanity.com/paper/2205.06975/) (2022)
- [Geometry-Aware Global Feature Aggregation for Real-Time Indirect Illumination](https://mlsanity.com/paper/2508.08826/) (2025)
- [IntrinsicReal: Adapting IntrinsicAnything from Synthetic to Real Objects](https://mlsanity.com/paper/2509.00777/) (2025)
- [GeRM: A Generative Rendering Model From Physically Realistic to Photorealistic](https://mlsanity.com/paper/2604.09304/) (2026)

## Abstract (arXiv)

We present a method to improve the visual realism of low-quality, synthetic images, e.g. OpenGL renderings. Training an unpaired synthetic-to-real translation network in image space is severely under-constrained and produces visible artifacts. Instead, we propose a semi-supervised approach that operates on the disentangled shading and albedo layers of the image. Our two-stage pipeline first learns to predict accurate shading in a supervised fashion using physically-based renderings as targets, and further increases the realism of the textures and shading with an improved CycleGAN network. Extensive evaluations on the SUNCG indoor scene dataset demonstrate that our approach yields more realistic images compared to other state-of-the-art approaches. Furthermore, networks trained on our generated "real" images predict more accurate depth and normals than domain adaptation approaches, suggesting that improving the visual realism of the images can be more effective than imposing task-specific losses.