# Generative View Synthesis: From Single-view Semantics to Novel-view Images

arXiv: https://arxiv.org/abs/2008.09106 · HTML: https://mlsanity.com/paper/2008.09106/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2020.

## Built on by (most cited first, 12 of 12)

- [StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D Mutual Learning](https://mlsanity.com/paper/2205.12183/) (2022)
- [Neural Trajectory Fields for Dynamic Novel View Synthesis](https://mlsanity.com/paper/2105.05994/) (2021)
- [Learning to Stylize Novel Views](https://mlsanity.com/paper/2105.13509/) (2021)
- [Generative Multiplane Images: Making a 2D GAN 3D-Aware](https://mlsanity.com/paper/2207.10642/) (2022)
- [UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene](https://mlsanity.com/paper/2208.07059/) (2022)
- [Neural Deformable Voxel Grid for Fast Optimization of Dynamic View Synthesis](https://mlsanity.com/paper/2206.07698/) (2022)
- [3D Scene Generation: A Survey](https://mlsanity.com/paper/2505.05474/) (2025)
- [Image Amodal Completion: A Survey](https://mlsanity.com/paper/2207.02062/) (2022)
- [Embedding Novel Views in a Single JPEG Image](https://mlsanity.com/paper/2108.13003/) (2021)
- [Painting 3D Nature in 2D: View Synthesis of Natural Scenes from a Single Semantic Mask](https://mlsanity.com/paper/2302.07224/) (2023)
- [Appearance Matching Adapter for Exemplar-based Semantic Image Synthesis in-the-Wild](https://mlsanity.com/paper/2412.03150/) (2024)
- [Enhancing Novel View Synthesis from extremely sparse views with SfM-free 3D Gaussian Splatting Framework](https://mlsanity.com/paper/2508.15457/) (2025)

## Abstract (arXiv)

Content creation, central to applications such as virtual reality, can be a tedious and time-consuming. Recent image synthesis methods simplify this task by offering tools to generate new views from as little as a single input image, or by converting a semantic map into a photorealistic image. We propose to push the envelope further, and introduce Generative View Synthesis (GVS), which can synthesize multiple photorealistic views of a scene given a single semantic map. We show that the sequential application of existing techniques, e.g., semantics-to-image translation followed by monocular view synthesis, fail at capturing the scene's structure. In contrast, we solve the semantics-to-image translation in concert with the estimation of the 3D layout of the scene, thus producing geometrically consistent novel views that preserve semantic structures. We first lift the input 2D semantic map onto a 3D layered representation of the scene in feature space, thereby preserving the semantic labels of 3D geometric structures. We then project the layered features onto the target views to generate the final novel-view images. We verify the strengths of our method and compare it with several advanced baselines on three different datasets. Our approach also allows for style manipulation and image editing operations, such as the addition or removal of objects, with simple manipulations of the input style images and semantic maps respectively. Visit the project page at https://gvsnet.github.io.