# Geometric Image Synthesis

arXiv: https://arxiv.org/abs/1809.04696 · HTML: https://mlsanity.com/paper/1809.04696/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Sep 2018.

## Built on by (most cited first, 16 of 16)

- [gradSim: Differentiable simulation for system identification and visuomotor control](https://mlsanity.com/paper/2104.02646/) (2021)
- [Enhancing Photorealism Enhancement](https://mlsanity.com/paper/2105.04619/) (2021)
- [A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis](https://mlsanity.com/paper/2110.15678/) (2021)
- [CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fields](https://mlsanity.com/paper/2103.17269/) (2021)
- [Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis](https://mlsanity.com/paper/2204.06307/) (2022)
- [VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relighting](https://mlsanity.com/paper/2201.04873/) (2022)
- [3D-Aware Indoor Scene Synthesis with Depth Priors](https://mlsanity.com/paper/2202.08553/) (2022)
- [Long-Term Temporally Consistent Unpaired Video Translation from Simulated Surgical 3D Data](https://mlsanity.com/paper/2103.17204/) (2021)
- [A Survey on Deep Generative 3D-aware Image Synthesis](https://mlsanity.com/paper/2210.14267/) (2022)
- [Designing a Hybrid Neural System to Learn Real-world Crack Segmentation from Fractal-based Simulation](https://mlsanity.com/paper/2309.09637/) (2023)
- [Unsupervised Novel View Synthesis from a Single Image](https://mlsanity.com/paper/2102.03285/) (2021)
- [Photo-realistic Neural Domain Randomization](https://mlsanity.com/paper/2210.12682/) (2022)
- [LANe: Lighting-Aware Neural Fields for Compositional Scene Synthesis](https://mlsanity.com/paper/2304.03280/) (2023)
- [Likelihood-Based Generative Radiance Field with Latent Space Energy-Based Model for 3D-Aware Disentangled Image Representation](https://mlsanity.com/paper/2304.07918/) (2023)
- [P2I-NET: Mapping Camera Pose to Image via Adversarial Learning for New View Synthesis in Real Indoor Environments](https://mlsanity.com/paper/2309.15526/) (2023)
- [Adversarial Appearance Learning in Augmented Cityscapes for Pedestrian Recognition in Autonomous Driving](https://mlsanity.com/paper/2509.13507/) (2025)

## Abstract (arXiv)

The task of generating natural images from 3D scenes has been a long standing goal in computer graphics. On the other hand, recent developments in deep neural networks allow for trainable models that can produce natural-looking images with little or no knowledge about the scene structure. While the generated images often consist of realistic looking local patterns, the overall structure of the generated images is often inconsistent. In this work we propose a trainable, geometry-aware image generation method that leverages various types of scene information, including geometry and segmentation, to create realistic looking natural images that match the desired scene structure. Our geometrically-consistent image synthesis method is a deep neural network, called Geometry to Image Synthesis (GIS) framework, which retains the advantages of a trainable method, e.g., differentiability and adaptiveness, but, at the same time, makes a step towards the generalizability, control and quality output of modern graphics rendering engines. We utilize the GIS framework to insert vehicles in outdoor driving scenes, as well as to generate novel views of objects from the Linemod dataset. We qualitatively show that our network is able to generalize beyond the training set to novel scene geometries, object shapes and segmentations. Furthermore, we quantitatively show that the GIS framework can be used to synthesize large amounts of training data which proves beneficial for training instance segmentation models.