mlsanityData to 7 Oct 2026

Learning A Locally Unified 3D Point Cloud for View Synthesis

arXiv 2209.05013 ↗First posted 12 Sep 2022cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

In this paper, we explore the problem of 3D point cloud representation-based view synthesis from a set of sparse source views. To tackle this challenging problem, we propose a new deep learning-based view synthesis paradigm that learns a locally unified 3D point cloud from source views. Specifically, we first construct sub-point clouds by projecting source views to 3D space based on their depth maps. Then, we learn the locally unified 3D point cloud by adaptively fusing points at a local neighborhood defined on the union of the sub-point clouds. Besides, we also propose a 3D geometry-guided image restoration module to fill the holes and recover high-frequency details of the rendered novel views. Experimental results on three benchmark datasets demonstrate that our method can improve the average PSNR by more than 4 dB while preserving more accurate visual details, compared with state-of-the-art view synthesis methods.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 44 of its 69 references that are in the map. First posted on arXiv 12 Sep 2022.

Builds on

44 of 69 references.

References · 44

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  4. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  5. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  6. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  7. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  8. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  9. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  10. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  11. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 584
  12. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
32 more
  1. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  2. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  3. 2019Neural Point-Based Graphicscited by 326
  4. 2019DeepView: View Synthesis with Learned Gradient Descentcited by 253
  5. 2020Free View Synthesiscited by 231
  6. 2018Stereo Magnification: Learning View Synthesis using Multiplane Imagescited by 206
  7. 20203D Photography using Context-aware Layered Depth Inpaintingcited by 206
  8. 2017Learning a Multi-View Stereo Machinecited by 194
  9. 2019Pushing the Boundaries of View Extrapolation with Multiplane Imagescited by 191
  10. 2021Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenescited by 190
  11. 2020GRF: Learning a General Radiance Field for 3D Representation and Renderingcited by 183
  12. 2019Neural Volumes: Learning Dynamic Renderable Volumes from Imagescited by 173
  13. 2016View Synthesis by Appearance Flowcited by 172
  14. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  15. 2020Stable View Synthesiscited by 165
  16. 2019Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Imagescited by 164
  17. 2019kPAM: KeyPoint Affordances for Category-Level Robotic Manipulationcited by 152
  18. 2021MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesiscited by 133
  19. 20193D Ken Burns Effect from a Single Imagecited by 128
  20. 2019Neural Rerendering in the Wildcited by 121
  21. 2016Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervisioncited by 106
  22. 2018Extreme View Synthesiscited by 94
  23. 2018Layer-structured 3D Scene Inference via View Synthesiscited by 82
  24. 2016Unsupervised Learning of 3D Structure from Imagescited by 67
  25. 2019Revealing Scenes by Inverting Structure from Motion Reconstructionscited by 66
  26. 2020Crowdsampling the Plenoptic Functioncited by 62
  27. 2019Neural Point Cloud Rendering via Multi-Plane Projectioncited by 60
  28. 2017Transformation-Grounded Image Generation Network for Novel 3D View Synthesiscited by 59
  29. 2022FWD: Real-time Novel View Synthesis with Forward Warping and Depthcited by 39
  30. 2019Object-Driven Multi-Layer Scene Decomposition From a Single Imagecited by 26
  31. 2021Self-Supervised Visibility Learning for Novel View Synthesiscited by 14
  32. 2020Novel View Synthesis from Single Images via Point Cloud Transformationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 1.

6 papers.

  1. 2024NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizercited by 79
  2. 2024MoDGS: Dynamic Gaussian Splatting from Casually-captured Monocular Videos with Depth Priorscited by 51
  3. 2024Rendering-Oriented 3D Point Cloud Attribute Compression using Sparse Tensor-based Transformercited by 4
  4. 2025Towards Integrating Multi-Spectral Imaging with Gaussian Splattingcited by 2
  5. 2025CloseUpShot: Close-up Novel View Synthesis from Sparse-views via Point-conditioned Diffusion Modelcited by 2
  6. 2026CGGS: Consistency-Augmented Geometric Gaussian Splatting for Ego-Centric 3D Scene Generationcited by 0