mlsanityData to 7 Oct 2026

Point2Pix: Photo-Realistic Point Cloud Rendering via Neural Radiance Fields

arXiv 2303.16482 ↗First posted 29 Mar 2023cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Synthesizing photo-realistic images from a point cloud is challenging because of the sparsity of point cloud representation. Recent Neural Radiance Fields and extensions are proposed to synthesize realistic images from 2D input. In this paper, we present Point2Pix as a novel point renderer to link the 3D sparse point clouds with 2D dense image pixels. Taking advantage of the point cloud 3D prior and NeRF rendering pipeline, our method can synthesize high-quality images from colored point clouds, generally for novel indoor scenes. To improve the efficiency of ray sampling, we propose point-guided sampling, which focuses on valid samples. Also, we present Point Encoding to build Multi-scale Radiance Fields that provide discriminative 3D point features. Finally, we propose Fusion Encoding to efficiently synthesize high-quality images. Extensive experiments on the ScanNet and ArkitScenes datasets demonstrate the effectiveness and generalization.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 41 of its 59 references that are in the map. First posted on arXiv 29 Mar 2023.

Builds on

41 of 59 references.

References · 41

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2016Layer Normalizationcited by 5,579
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  7. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  8. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  9. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  10. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  11. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  12. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
29 more
  1. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  2. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  3. 2020Neural Sparse Voxel Fieldscited by 959
  4. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  5. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  6. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  7. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  8. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  9. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  10. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  11. 2018Open3D: A Modern Library for 3D Data Processingcited by 717
  12. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  13. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  14. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  15. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  16. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  17. 2021Neural RGB-D Surface Reconstructioncited by 312
  18. 2021DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networkscited by 232
  19. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  20. 2017Submanifold Sparse Convolutional Networkscited by 223
  21. 2018SPLATNet: Sparse Lattice Networks for Point Cloud Processingcited by 158
  22. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
  23. 2021Animatable Neural Radiance Fields from Monocular RGB Videoscited by 109
  24. 2021Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Controlcited by 106
  25. 2015Sparse 3D convolutional neural networkscited by 71
  26. 2018Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Cloudscited by 62
  27. 2019Neural Point Cloud Rendering via Multi-Plane Projectioncited by 60
  28. 2016Deep Outdoor Illumination Estimationcited by 53
  29. 2021ICM-3D: Instantiated Category Modeling for 3D Instance Segmentationcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 4; 2026: 1.

12 papers.

  1. 2024LidaRF: Delving into Lidar for Neural Radiance Field on Street Scenescited by 14
  2. 2025RoGSplat: Learning Robust Generalizable Human Gaussian Splatting from Sparse Multi-View Imagescited by 12
  3. 2024X-Ray: A Sequential 3D Representation For Generationcited by 9
  4. 2024Learning Robust Generalizable Radiance Field with Visibility and Feature Augmented Point Representationcited by 7
  5. 2024HashPoint: Accelerated Point Searching and Sampling for Neural Renderingcited by 4
  6. 2025Real-time Neural Rendering of LiDAR Point Cloudscited by 1
  7. 2025Sparse Point Cloud Patches Rendering via Splitting 2D Gaussianscited by 1
  8. 2024Methods and strategies for improving the novel view synthesis quality of neural radiation fieldcited by 0
  9. 2024GPN: Generative Point-based NeRFcited by 0
  10. 2024NARVis: Neural Accelerated Rendering for Real-Time Scientific Point Cloud Visualizationcited by 0
  11. 2025CRISTAL: Real-time Camera Registration in Static LiDAR Scans using Neural Renderingcited by 0
  12. 2026Bridging the Dimensionality Gap: A Taxonomy and Survey of 2D Vision Model Adaptation for 3D Analysiscited by 0