# Single-View View Synthesis with Self-Rectified Pseudo-Stereo

arXiv: https://arxiv.org/abs/2304.09527 · HTML: https://mlsanity.com/paper/2304.09527/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 57 references that are in the map.
- First posted on arXiv 19 Apr 2023.

## Builds on (most cited first, 30 of 39)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Neural Point-Based Graphics](https://mlsanity.com/paper/1906.08240/) (2019)
- [Single-View View Synthesis with Multiplane Images](https://mlsanity.com/paper/2004.11364/) (2020)
- [DeepView: View Synthesis with Learned Gradient Descent](https://mlsanity.com/paper/1906.07316/) (2019)
- [3D Photography using Context-aware Layered Depth Inpainting](https://mlsanity.com/paper/2004.04727/) (2020)
- [Pushing the Boundaries of View Extrapolation with Multiplane Images](https://mlsanity.com/paper/1905.00413/) (2019)
- [View Synthesis by Appearance Flow](https://mlsanity.com/paper/1605.03557/) (2016)
- [What Do Compressed Deep Neural Networks Forget?](https://mlsanity.com/paper/1911.05248/) (2019)
- [MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis](https://mlsanity.com/paper/2103.14910/) (2021)
- [Deep Convolutional Inverse Graphics Network](https://mlsanity.com/paper/1503.03167/) (2015)
- [Neural Rerendering in the Wild](https://mlsanity.com/paper/1904.04290/) (2019)
- [Extreme View Synthesis](https://mlsanity.com/paper/1812.04777/) (2018)
- [Layer-structured 3D Scene Inference via View Synthesis](https://mlsanity.com/paper/1807.10264/) (2018)
- [Deep3D: Fully Automatic 2D-to-3D Video Conversion with Deep Convolutional Neural Networks](https://mlsanity.com/paper/1604.03650/) (2016)
- [VisemeNet: Audio-Driven Animator-Centric Speech Animation](https://mlsanity.com/paper/1805.09488/) (2018)
- [Learning to Synthesize a 4D RGBD Light Field from a Single Image](https://mlsanity.com/paper/1708.03292/) (2017)
- [Transformation-Grounded Image Generation Network for Novel 3D View Synthesis](https://mlsanity.com/paper/1703.02921/) (2017)
- [Multi-view 3D Models from Single Images with a Convolutional Network](https://mlsanity.com/paper/1511.06702/) (2015)
- [Forget About the LiDAR: Self-Supervised Depth Estimators with MED Probability Volumes](https://mlsanity.com/paper/2008.03633/) (2020)
- [Self-Damaging Contrastive Learning](https://mlsanity.com/paper/2106.02990/) (2021)

## Built on by (most cited first, 4 of 4)

- [Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities](https://mlsanity.com/paper/2401.08045/) (2024)
- [Learning with Unreliability: Fast Few-shot Voxel Radiance Fields with Relative Geometric Consistency](https://mlsanity.com/paper/2403.17638/) (2024)
- [OPONeRF: One-Point-One NeRF for Robust Neural Rendering](https://mlsanity.com/paper/2409.20043/) (2024)
- [CheapNVS: Real-Time On-Device Narrow-Baseline Novel View Synthesis](https://mlsanity.com/paper/2501.14533/) (2025)

## Abstract (arXiv)

Synthesizing novel views from a single view image is a highly ill-posed problem. We discover an effective solution to reduce the learning ambiguity by expanding the single-view view synthesis problem to a multi-view setting. Specifically, we leverage the reliable and explicit stereo prior to generate a pseudo-stereo viewpoint, which serves as an auxiliary input to construct the 3D space. In this way, the challenging novel view synthesis process is decoupled into two simpler problems of stereo synthesis and 3D reconstruction. In order to synthesize a structurally correct and detail-preserved stereo image, we propose a self-rectified stereo synthesis to amend erroneous regions in an identify-rectify manner. Hard-to-train and incorrect warping samples are first discovered by two strategies, 1) pruning the network to reveal low-confident predictions; and 2) bidirectionally matching between stereo images to allow the discovery of improper mapping. These regions are then inpainted to form the final pseudo-stereo. With the aid of this extra input, a preferable 3D reconstruction can be easily obtained, and our method can work with arbitrary 3D representations. Extensive experiments show that our method outperforms state-of-the-art single-view view synthesis methods and stereo synthesis methods.