# Deep View Synthesis via Self-Consistent Generative Network

arXiv: https://arxiv.org/abs/2101.10844 · HTML: https://mlsanity.com/paper/2101.10844/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 51 references that are in the map.
- First posted on arXiv 19 Jan 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [DeepView: View Synthesis with Learned Gradient Descent](https://mlsanity.com/paper/1906.07316/) (2019)
- [Stereo Magnification: Learning View Synthesis using Multiplane Images](https://mlsanity.com/paper/1805.09817/) (2018)
- [DeepStereo: Learning to Predict New Views from the World's Imagery](https://mlsanity.com/paper/1506.06825/) (2015)
- [View Synthesis by Appearance Flow](https://mlsanity.com/paper/1605.03557/) (2016)
- [Extreme View Synthesis](https://mlsanity.com/paper/1812.04777/) (2018)
- [Closed-loop Matters: Dual Regression Networks for Single Image Super-Resolution](https://mlsanity.com/paper/2003.07018/) (2020)
- [Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis](https://mlsanity.com/paper/1704.04086/) (2017)
- [Transformation-Grounded Image Generation Network for Novel 3D View Synthesis](https://mlsanity.com/paper/1703.02921/) (2017)
- [Multi-view 3D Models from Single Images with a Convolutional Network](https://mlsanity.com/paper/1511.06702/) (2015)
- [Learning to Generate Chairs, Tables and Cars with Convolutional Networks](https://mlsanity.com/paper/1411.5928/) (2014)
- [Geometry-aware Deep Network for Single-Image Novel View Synthesis](https://mlsanity.com/paper/1804.06008/) (2018)
- [Auto-Embedding Generative Adversarial Networks for High Resolution Image Synthesis](https://mlsanity.com/paper/1903.11250/) (2019)
- [Improving Generative Adversarial Networks with Local Coordinate Coding](https://mlsanity.com/paper/2008.00942/) (2020)
- [Fast View Synthesis with Deep Stereo Vision](https://mlsanity.com/paper/1804.09690/) (2018)

## Built on by (most cited first, 10 of 10)

- [SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Grounding](https://mlsanity.com/paper/2412.04383/) (2024)
- [One-Shot Diffusion Mimicker for Handwritten Text Generation](https://mlsanity.com/paper/2409.04004/) (2024)
- [AGTGAN: Unpaired Image Translation for Photographic Ancient Character Generation](https://mlsanity.com/paper/2303.07012/) (2023)
- [CBARF: Cascaded Bundle-Adjusting Neural Radiance Fields from Imperfect Camera Poses](https://mlsanity.com/paper/2310.09776/) (2023)
- [Beyond Isolated Words: Diffusion Brush for Handwritten Text-Line Generation](https://mlsanity.com/paper/2508.03256/) (2025)
- [Boosting Semi-Supervised Learning with Contrastive Complementary Labeling](https://mlsanity.com/paper/2212.06643/) (2022)
- [SceneCompleter: Dense 3D Scene Completion for Generative Novel View Synthesis](https://mlsanity.com/paper/2506.10981/) (2025)
- [See360: Novel Panoramic View Interpolation](https://mlsanity.com/paper/2401.03431/) (2024)
- [AccidentSim: Generating Vehicle Collision Videos with Physically Realistic Collision Trajectories from Real-World Accident Reports](https://mlsanity.com/paper/2503.20654/) (2025)
- [UniGround: Universal 3D Visual Grounding via Training-Free Scene Parsing](https://mlsanity.com/paper/2603.08131/) (2026)

## Abstract (arXiv)

View synthesis aims to produce unseen views from a set of views captured by two or more cameras at different positions. This task is non-trivial since it is hard to conduct pixel-level matching among different views. To address this issue, most existing methods seek to exploit the geometric information to match pixels. However, when the distinct cameras have a large baseline (i.e., far away from each other), severe geometry distortion issues would occur and the geometric information may fail to provide useful guidance, resulting in very blurry synthesized images. To address the above issues, in this paper, we propose a novel deep generative model, called Self-Consistent Generative Network (SCGN), which synthesizes novel views from the given input views without explicitly exploiting the geometric information. The proposed SCGN model consists of two main components, i.e., a View Synthesis Network (VSN) and a View Decomposition Network (VDN), both employing an Encoder-Decoder structure. Here, the VDN seeks to reconstruct input views from the synthesized novel view to preserve the consistency of view synthesis. Thanks to VDN, SCGN is able to synthesize novel views without using any geometric rectification before encoding, making it easier for both training and applications. Finally, adversarial loss is introduced to improve the photo-realism of novel views. Both qualitative and quantitative comparisons against several state-of-the-art methods on two benchmark tasks demonstrated the superiority of our approach.