mlsanityData to 7 Oct 2026

Points2NeRF: Generating Neural Radiance Fields from 3D point cloud

arXiv 2206.01290 ↗First posted 2 Jun 2022cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Contemporary registration devices for 3D visual information, such as LIDARs and various depth cameras, capture data as 3D point clouds. In turn, such clouds are challenging to be processed due to their size and complexity. Existing methods address this problem by fitting a mesh to the point cloud and rendering it instead. This approach, however, leads to the reduced fidelity of the resulting visualization and misses color information of the objects crucial in computer graphics applications. In this work, we propose to mitigate this challenge by representing 3D objects as Neural Radiance Fields (NeRFs). We leverage a hypernetwork paradigm and train the model to take a 3D point cloud with the associated color values and return a NeRF network's weights that reconstruct 3D objects from input 2D images. Our method provides efficient 3D object representation and offers several advantages over the existing approaches, including the ability to condition NeRFs and improved generalization beyond objects seen in training. The latter we also confirmed in the results of our empirical evaluation.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 53 of its 72 references that are in the map. First posted on arXiv 2 Jun 2022.

Builds on

53 of 72 references.

References · 53

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  5. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  6. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  7. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  8. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  9. 2018On the Spectral Bias of Neural Networkscited by 1,321
  10. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  11. 2016HyperNetworkscited by 1,003
  12. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
38 more
  1. 2020Neural Sparse Voxel Fieldscited by 959
  2. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  3. 2020Convolutional Occupancy Networkscited by 760
  4. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  5. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  6. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  7. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  8. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  9. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  10. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  11. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 584
  12. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 564
  13. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  14. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  15. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  16. 2017Wasserstein Auto-Encoderscited by 371
  17. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  18. 2019Continual learning with hypernetworkscited by 267
  19. 2020Learning Gradient Fields for Shape Generationcited by 245
  20. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  21. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  22. 2017Learning Efficient Point Cloud Generation for Dense 3D Object Reconstructioncited by 133
  23. 2021NeRF-VAE: A Geometry Aware 3D Scene Generative Modelcited by 111
  24. 2022POCO: Point Convolution for Surface Reconstructioncited by 91
  25. 2017Hierarchical Surface Prediction for 3D Object Reconstructioncited by 89
  26. 2021Generative Models as Distributions of Functionscited by 89
  27. 2021CoNeRF: Controllable Neural Radiance Fieldscited by 65
  28. 2021SA-ConvONet: Sign-Agnostic Optimization of Convolutional Occupancy Networkscited by 59
  29. 2021Neural Point Light Fieldscited by 51
  30. 2017OctNetFusion: Learning Depth Fusion from Datacited by 45
  31. 2017SurfNet: Generating 3D shape surfaces using deep residual networkscited by 32
  32. 2018SCORES: Shape Composition with Recursive Substructure Priorscited by 23
  33. 2022Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Imagescited by 21
  34. 2020Hypernetwork approach to generating point cloudscited by 20
  35. 2021HyperPocket: Generative Point Cloud Completioncited by 12
  36. 2018Cramer-Wold AutoEncodercited by 10
  37. 2017Stochastic Maximum Likelihood Optimization via Hypernetworkscited by 7
  38. 2021Continuous Conditional Random Field Convolution for Point Cloud Segmentationcited by 7

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 7; 2024: 1; 2025: 1.

11 papers.

  1. 2023Neural Point-based Volumetric Avatar: Surface-guided Neural Points for Efficient and Photorealistic Volumetric Head Avatarcited by 14
  2. 2022RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Controlcited by 10
  3. 2023Targeted Adversarial Attacks on Generalizable Neural Radiance Fieldscited by 10
  4. 2023CaesarNeRF: Calibrated Semantic Representation for Few-shot Generalizable Neural Renderingcited by 10
  5. 2023HyperNeRFGAN: Hypernetwork approach to 3D NeRF GANcited by 7
  6. 2023InsertNeRF: Instilling Generalizability into NeRF with HyperNet Modulescited by 7
  7. 2023MultiPlaneNeRF: Neural Radiance Field with Non-Trainable Representationcited by 3
  8. 2024MergeNet: Explicit Mesh Reconstruction from Sparse Point Clouds via Edge Predictioncited by 2
  9. 2022Deep Generative Models on 3D Representations: A Surveycited by 1
  10. 2023NeRFlame: FLAME-based conditioning of NeRF for 3D face renderingcited by 1
  11. 2025ConsistentDreamer: View-Consistent Meshes Through Balanced Multi-View Gaussian Optimizationcited by 0