# Neural Point Cloud Rendering via Multi-Plane Projection

arXiv: https://arxiv.org/abs/1912.04645 · HTML: https://mlsanity.com/paper/1912.04645/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2019.

## Built on by (most cited first, 30 of 60)

- [SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes](https://mlsanity.com/paper/2312.14937/) (2023)
- [Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering](https://mlsanity.com/paper/2109.01847/) (2021)
- [Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splatting](https://mlsanity.com/paper/2401.01339/) (2024)
- [NeRF-Editing: Geometry Editing of Neural Radiance Fields](https://mlsanity.com/paper/2205.04978/) (2022)
- [PLA: Language-Driven Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2211.16312/) (2022)
- [Unconstrained Scene Generation with Locally Conditioned Radiance Fields](https://mlsanity.com/paper/2104.00670/) (2021)
- [Hallucinated Neural Radiance Fields in the Wild](https://mlsanity.com/paper/2111.15246/) (2021)
- [Transfer Learning from Synthetic to Real LiDAR Point Cloud for Semantic Segmentation](https://mlsanity.com/paper/2107.05399/) (2021)
- [MultiDiff: Consistent Novel View Synthesis from a Single Image](https://mlsanity.com/paper/2406.18524/) (2024)
- [Advances in 3D Generation: A Survey](https://mlsanity.com/paper/2401.17807/) (2024)
- [Direct-PoseNet: Absolute Pose Regression with Photometric Consistency](https://mlsanity.com/paper/2104.04073/) (2021)
- [Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Images](https://mlsanity.com/paper/2205.11733/) (2022)
- [Style-based Point Generator with Adversarial Rendering for Point Cloud Completion](https://mlsanity.com/paper/2103.02535/) (2021)
- [Grad-PU: Arbitrary-Scale Point Cloud Upsampling via Gradient Descent with Learned Distance Functions](https://mlsanity.com/paper/2304.11846/) (2023)
- [Fast Learning Radiance Fields by Shooting Much Fewer Rays](https://mlsanity.com/paper/2208.06821/) (2022)
- [Lowis3D: Language-Driven Open-World Instance-Level 3D Scene Understanding](https://mlsanity.com/paper/2308.00353/) (2023)
- [Fast and Explicit Neural View Synthesis](https://mlsanity.com/paper/2107.05775/) (2021)
- [Pointersect: Neural Rendering with Cloud-Ray Intersection](https://mlsanity.com/paper/2304.12390/) (2023)
- [SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrix](https://mlsanity.com/paper/2407.00367/) (2024)
- [Learning Continuous Implicit Field with Local Distance Indicator for Arbitrary-Scale Point Cloud Upsampling](https://mlsanity.com/paper/2312.15133/) (2023)
- [Known Operator Learning and Hybrid Machine Learning in Medical Imaging -- A Review of the Past, the Present, and the Future](https://mlsanity.com/paper/2108.04543/) (2021)
- [Hybrid Neural Rendering for Large-Scale Scenes with Motion Blur](https://mlsanity.com/paper/2304.12652/) (2023)
- [Seal-3D: Interactive Pixel-Level Editing for Neural Radiance Fields](https://mlsanity.com/paper/2307.15131/) (2023)
- [Deformable Radial Kernel Splatting](https://mlsanity.com/paper/2412.11752/) (2024)
- [TriVol: Point Cloud Rendering via Triple Volumes](https://mlsanity.com/paper/2303.16485/) (2023)
- [NDF: Neural Deformable Fields for Dynamic Human Modelling](https://mlsanity.com/paper/2207.09193/) (2022)
- [Reconstructing 4D Spatial Intelligence: A Survey](https://mlsanity.com/paper/2507.21045/) (2025)
- [Point2Pix: Photo-Realistic Point Cloud Rendering via Neural Radiance Fields](https://mlsanity.com/paper/2303.16482/) (2023)
- [Grounding Creativity in Physics: A Brief Survey of Physical Priors in AIGC](https://mlsanity.com/paper/2502.07007/) (2025)
- [Boosting Point Clouds Rendering via Radiance Mapping](https://mlsanity.com/paper/2210.15107/) (2022)

## Abstract (arXiv)

We present a new deep point cloud rendering pipeline through multi-plane projections. The input to the network is the raw point cloud of a scene and the output are image or image sequences from a novel view or along a novel camera trajectory. Unlike previous approaches that directly project features from 3D points onto 2D image domain, we propose to project these features into a layered volume of camera frustum. In this way, the visibility of 3D points can be automatically learnt by the network, such that ghosting effects due to false visibility check as well as occlusions caused by noise interferences are both avoided successfully. Next, the 3D feature volume is fed into a 3D CNN to produce multiple layers of images w.r.t. the space division in the depth directions. The layered images are then blended based on learned weights to produce the final rendering results. Experiments show that our network produces more stable renderings compared to previous methods, especially near the object boundaries. Moreover, our pipeline is robust to noisy and relatively sparse point cloud for a variety of challenging scenes.