# Enhanced Stable View Synthesis

arXiv: https://arxiv.org/abs/2303.17094 · HTML: https://mlsanity.com/paper/2303.17094/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 54 references that are in the map.
- First posted on arXiv 30 Mar 2023.

## Builds on (most cited first, 25 of 25)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans](https://mlsanity.com/paper/2110.04994/) (2021)
- [State of the Art on Neural Rendering](https://mlsanity.com/paper/2004.03805/) (2020)
- [Free View Synthesis](https://mlsanity.com/paper/2008.05511/) (2020)
- [PatchmatchNet: Learned Multi-View Patchmatch Stereo](https://mlsanity.com/paper/2012.01411/) (2020)
- [Self-Calibrating Neural Radiance Fields](https://mlsanity.com/paper/2108.13826/) (2021)
- [Point-Based Neural Rendering with Per-View Optimization](https://mlsanity.com/paper/2109.02369/) (2021)
- [Stable View Synthesis](https://mlsanity.com/paper/2011.07233/) (2020)
- [A Survey of Structure from Motion](https://mlsanity.com/paper/1701.08493/) (2017)
- [Deep Multimodal Fusion by Channel Exchanging](https://mlsanity.com/paper/2011.05005/) (2020)
- [Enhancing Photorealism Enhancement](https://mlsanity.com/paper/2105.04619/) (2021)
- [VA-DepthNet: A Variational Approach to Single Image Depth Prediction](https://mlsanity.com/paper/2302.06556/) (2023)
- [NeuRoRA: Neural Robust Rotation Averaging](https://mlsanity.com/paper/1912.04485/) (2019)
- [Monocular Dense 3D Reconstruction of a Complex Dynamic Scene from Two Perspective Frames](https://mlsanity.com/paper/1708.04398/) (2017)
- [Superpixel Soup: Monocular Dense 3D Reconstruction of a Complex Dynamic Scene](https://mlsanity.com/paper/1911.09092/) (2019)
- [Correlational Neural Networks](https://mlsanity.com/paper/1504.07225/) (2015)
- [Robustifying the Multi-Scale Representation of Neural Radiance Fields](https://mlsanity.com/paper/2210.04233/) (2022)

## Built on by (most cited first, 5 of 5)

- [Taming 3DGS: High-Quality Radiance Fields with Limited Resources](https://mlsanity.com/paper/2406.15643/) (2024)
- [InsertNeRF: Instilling Generalizability into NeRF with HyperNet Modules](https://mlsanity.com/paper/2308.13897/) (2023)
- [Stereo Risk: A Continuous Modeling Approach to Stereo Matching](https://mlsanity.com/paper/2407.03152/) (2024)
- [Rethinking Dense Optical Flow without Test-Time Scaling](https://mlsanity.com/paper/2605.08000/) (2026)
- [3D Gaussian Splatting for Efficient Retrospective Dynamic Scene Novel View Synthesis with a Standardized Benchmark](https://mlsanity.com/paper/2605.12437/) (2026)

## Abstract (arXiv)

We introduce an approach to enhance the novel view synthesis from images taken from a freely moving camera. The introduced approach focuses on outdoor scenes where recovering accurate geometric scaffold and camera pose is challenging, leading to inferior results using the state-of-the-art stable view synthesis (SVS) method. SVS and related methods fail for outdoor scenes primarily due to (i) over-relying on the multiview stereo (MVS) for geometric scaffold recovery and (ii) assuming COLMAP computed camera poses as the best possible estimates, despite it being well-studied that MVS 3D reconstruction accuracy is limited to scene disparity and camera-pose accuracy is sensitive to key-point correspondence selection. This work proposes a principled way to enhance novel view synthesis solutions drawing inspiration from the basics of multiple view geometry. By leveraging the complementary behavior of MVS and monocular depth, we arrive at a better scene depth per view for nearby and far points, respectively. Moreover, our approach jointly refines camera poses with image-based rendering via multiple rotation averaging graph optimization. The recovered scene depth and the camera-pose help better view-dependent on-surface feature aggregation of the entire scene. Extensive evaluation of our approach on the popular benchmark dataset, such as Tanks and Temples, shows substantial improvement in view synthesis results compared to the prior art. For instance, our method shows 1.5 dB of PSNR improvement on the Tank and Temples. Similar statistics are observed when tested on other benchmark datasets such as FVS, Mip-NeRF 360, and DTU.