# SynSin: End-to-end View Synthesis from a Single Image

arXiv: https://arxiv.org/abs/1912.08804 · HTML: https://mlsanity.com/paper/1912.08804/

## Key facts

- Cited by 364 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Dec 2019.

## Built on by (most cited first, 30 of 364)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [2D Gaussian Splatting for Geometrically Accurate Radiance Fields](https://mlsanity.com/paper/2403.17888/) (2024)
- [Depth-supervised NeRF: Fewer Views and Faster Training for Free](https://mlsanity.com/paper/2107.02791/) (2021)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction](https://mlsanity.com/paper/2312.12337/) (2023)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives](https://mlsanity.com/paper/2311.18259/) (2023)
- [Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis](https://mlsanity.com/paper/2104.00677/) (2021)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis](https://mlsanity.com/paper/2409.02048/) (2024)
- [Compact 3D Gaussian Representation for Radiance Field](https://mlsanity.com/paper/2311.13681/) (2023)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [Splatter Image: Ultra-Fast Single-View 3D Reconstruction](https://mlsanity.com/paper/2312.13150/) (2023)
- [GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting](https://mlsanity.com/paper/2404.19702/) (2024)
- [Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis](https://mlsanity.com/paper/2312.16812/) (2023)
- [Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior](https://mlsanity.com/paper/2303.14184/) (2023)
- [Mixture of Volumetric Primitives for Efficient Neural Rendering](https://mlsanity.com/paper/2103.01954/) (2021)
- [Generative Novel View Synthesis with 3D-Aware Diffusion Models](https://mlsanity.com/paper/2304.02602/) (2023)
- [Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models](https://mlsanity.com/paper/2303.11989/) (2023)
- [A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose](https://mlsanity.com/paper/2102.06199/) (2021)
- [A Survey on 3D Gaussian Splatting](https://mlsanity.com/paper/2401.03890/) (2024)
- [NeX: Real-time View Synthesis with Neural Basis Expansion](https://mlsanity.com/paper/2103.05606/) (2021)
- [WonderWorld: Interactive 3D Scene Generation from a Single Image](https://mlsanity.com/paper/2406.09394/) (2024)
- [Robust Dynamic Radiance Fields](https://mlsanity.com/paper/2301.02239/) (2023)
- [CityGaussian: Real-time High-quality Large-Scale Scene Rendering with Gaussians](https://mlsanity.com/paper/2404.01133/) (2024)

## Abstract (arXiv)

Single image view synthesis allows for the generation of new views of a scene given a single input image. This is challenging, as it requires comprehensively understanding the 3D scene from a single image. As a result, current methods typically use multiple images, train on ground-truth depth, or are limited to synthetic data. We propose a novel end-to-end model for this task; it is trained on real images without any ground-truth 3D information. To this end, we introduce a novel differentiable point cloud renderer that is used to transform a latent 3D point cloud of features into the target view. The projected features are decoded by our refinement network to inpaint missing regions and generate a realistic output image. The 3D component inside of our generative model allows for interpretable manipulation of the latent feature space at test time, e.g. we can animate trajectories from a single image. Unlike prior work, we can generate high resolution images and generalise to other input resolutions. We outperform baselines and prior work on the Matterport, Replica, and RealEstate10K datasets.