# Novel View Synthesis from a Single RGBD Image for Indoor Scenes

arXiv: https://arxiv.org/abs/2311.01065 · HTML: https://mlsanity.com/paper/2311.01065/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 2 Nov 2023.

## Builds on (most cited first, 12 of 12)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Geometry-Free View Synthesis: Transformers and no 3D Priors](https://mlsanity.com/paper/2104.07652/) (2021)
- [Region-based Quality Estimation Network for Large-scale Person Re-identification](https://mlsanity.com/paper/1711.08766/) (2017)
- [Multi-View Image Generation from a Single-View](https://mlsanity.com/paper/1704.04886/) (2017)
- [Generative Adversarial Frontal View to Bird View Synthesis](https://mlsanity.com/paper/1808.00327/) (2018)
- [Point Set Voting for Partial Point Cloud Analysis](https://mlsanity.com/paper/2007.04537/) (2020)
- [Single Image 3D Interpreter Network](https://mlsanity.com/paper/1604.08685/) (2016)
- [Impression Network for Video Object Detection](https://mlsanity.com/paper/1712.05896/) (2017)
- [Improving 3D Object Detection for Pedestrians with Virtual Multi-View Synthesis Orientation Estimation](https://mlsanity.com/paper/1907.06777/) (2019)
- [Depth-wise Decomposition for Accelerating Separable Convolutions in Efficient Convolutional Neural Networks](https://mlsanity.com/paper/1910.09455/) (2019)
- [Equivariant Map and Agent Geometry for Autonomous Driving Motion Prediction](https://mlsanity.com/paper/2310.13922/) (2023)

## Built on by (most cited first, 4 of 4)

- [UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Driving](https://mlsanity.com/paper/2503.24381/) (2025)
- [Objaverse++: Curated 3D Object Dataset with Quality Annotations](https://mlsanity.com/paper/2504.07334/) (2025)
- [RDD: Retrieval-Based Demonstration Decomposer for Planner Alignment in Long-Horizon Tasks](https://mlsanity.com/paper/2510.14968/) (2025)
- [EucliDreamer: Fast and High-Quality Texturing for 3D Models with Depth-Conditioned Stable Diffusion](https://mlsanity.com/paper/2404.10279/) (2024)

## Abstract (arXiv)

In this paper, we propose an approach for synthesizing novel view images from a single RGBD (Red Green Blue-Depth) input. Novel view synthesis (NVS) is an interesting computer vision task with extensive applications. Methods using multiple images has been well-studied, exemplary ones include training scene-specific Neural Radiance Fields (NeRF), or leveraging multi-view stereo (MVS) and 3D rendering pipelines. However, both are either computationally intensive or non-generalizable across different scenes, limiting their practical value. Conversely, the depth information embedded in RGBD images unlocks 3D potential from a singular view, simplifying NVS. The widespread availability of compact, affordable stereo cameras, and even LiDARs in contemporary devices like smartphones, makes capturing RGBD images more accessible than ever. In our method, we convert an RGBD image into a point cloud and render it from a different viewpoint, then formulate the NVS task into an image translation problem. We leveraged generative adversarial networks to style-transfer the rendered image, achieving a result similar to a photograph taken from the new perspective. We explore both unsupervised learning using CycleGAN and supervised learning with Pix2Pix, and demonstrate the qualitative results. Our method circumvents the limitations of traditional multi-image techniques, holding significant promise for practical, real-time applications in NVS.