mlsanityData to 14 Oct 2026

Ponder: Point Cloud Pre-training via Neural Rendering

arXiv 2301.00157 ↗First posted 31 Dec 2022cs.CVcited by 50

Not on the map. Why?

Abstract (arXiv)

We propose a novel approach to self-supervised learning of point cloud representations by differentiable neural rendering. Motivated by the fact that informative point cloud features should be able to encode rich geometry and appearance cues and render realistic images, we train a point-cloud encoder within a devised point-based neural renderer by comparing the rendered images with real images on massive RGB-D data. The learned point-cloud encoder can be easily integrated into various downstream tasks, including not only high-level tasks like 3D detection and segmentation, but low-level tasks like 3D reconstruction and image synthesis. Extensive experiments on various tasks demonstrate the superiority of our approach compared to existing pre-training methods.

Key facts. Cited by 50 papers in the map (data through 14 Oct 2026). Builds on 60 of its 73 references that are in the map. First posted on arXiv 31 Dec 2022.

Builds on

60 of 73 references.

References · 60

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2017Mask R-CNNcited by 7,277
  8. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  9. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  10. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  11. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  12. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
38 more
  1. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  2. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  3. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  4. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  5. 2017Deformable Convolutional Networkscited by 1,488
  6. 2020Point Transformercited by 1,470
  7. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  8. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  9. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
  10. 2020Neural Sparse Voxel Fieldscited by 959
  11. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 905
  12. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 861
  13. 2020Convolutional Occupancy Networkscited by 761
  14. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
  15. 2020Implicit Geometric Regularization for Learning Shapescited by 710
  16. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
  17. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 673
  18. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
  19. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  20. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 646
  21. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 504
  22. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  23. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 459
  24. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 429
  25. 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
  26. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  27. 2019Neural Point-Based Graphicscited by 327
  28. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 264
  29. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 260
  30. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  31. 2020Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contextscited by 210
  32. 2022Unifying Voxel-based Representation with Transformer for 3D Object Detectioncited by 202
  33. 2022CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understandingcited by 198
  34. 2020Unsupervised Point Cloud Pre-Training via Occlusion Completioncited by 186
  35. 2021Spatio-temporal Self-Supervised Representation Learning for 3D Point Cloudscited by 180
  36. 2022SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Viewscited by 177
  37. 2022Masked Discrimination for Self-Supervised Learning on Point Cloudscited by 155
  38. 2020H3DNet: 3D Object Detection Using Hybrid Geometric Primitivescited by 123

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 12; 2024: 20; 2025: 13; 2026: 5.

50 papers.

Sort:

By area:

  1. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 587
  2. 2024Grounded 3D-LLM with Referent Tokenscited by 115
  3. 2023CLIP-FO3D: Learning Free Open-world 3D Scene Representations from 2D Dense CLIPcited by 89
  4. 2023UniPAD: A Universal Pre-training Paradigm for Autonomous Drivingcited by 73
  5. 2023PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigmcited by 56
  6. 2023Point Cloud Pre-training with Diffusion Modelscited by 47
  7. 2024Point Cloud Matters: Rethinking the Impact of Different Observation Spaces on Robot Learningcited by 45
  8. 2023SAMPro3D: Locating SAM Prompts in 3D for Zero-Shot Instance Segmentationcited by 44
  9. 2024Learning Generalizable Feature Fields for Mobile Manipulationcited by 31
  10. 2024SPA: 3D Spatial-Awareness Enables Effective Embodied Representationcited by 28
  11. 2024SUGAR: Pre-training 3D Visual Representations for Roboticscited by 26
  12. 2024EFM3D: A Benchmark for Measuring Progress Towards 3D Egocentric Foundation Modelscited by 26
38 more
  1. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunitiescited by 24
  2. 2023Hulk: A Universal Knowledge Translator for Human-Centric Taskscited by 22
  3. 2023Self-supervised Learning for Pre-Training 3D Point Clouds: A Surveycited by 21
  4. 2025EVolSplat: Efficient Volume-based Gaussian Splatting for Urban View Synthesiscited by 21
  5. 2024NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D Representation Learning for Neural Radiance Fieldscited by 20
  6. 2024EvGGS: A Collaborative Learning Framework for Event-based Generalizable Gaussian Splattingcited by 20
  7. 2023Sculpting Holistic 3D Representation in Contrastive Language-Image-3D Pre-trainingcited by 18
  8. 2024Bringing Masked Autoencoders Explicit Contrastive Properties for Point Cloud Self-Supervised Learningcited by 16
  9. 2024Lift3D: Zero-Shot Lifting of Any 2D Vision Model to 3Dcited by 15
  10. 2024A Unified Framework for 3D Scene Understandingcited by 15
  11. 2023Point-GCC: Universal Self-supervised 3D Scene Pre-training via Geometry-Color Contrastcited by 14
  12. 2023Applications of Large Scale Foundation Models for Autonomous Drivingcited by 14
  13. 2024CooPre: Cooperative Pretraining for V2X Cooperative Perceptioncited by 11
  14. 2025PointSea: Point Cloud Completion via Self-structure Augmentationcited by 9
  15. 2024MIM4D: Masked Modeling with Multi-View Video for Autonomous Driving Representation Learningcited by 8
  16. 2023PRED: Pre-training via Semantic Rendering on LiDAR Point Cloudscited by 7
  17. 2024Self-supervised Learning of LiDAR 3D Point Clouds via 2D-3D Neural Calibrationcited by 7
  18. 2024Efficient Depth-Guided Urban View Synthesiscited by 7
  19. 2025Beyond One Shot, Beyond One Perspective: Cross-View and Long-Horizon Distillation for Better LiDAR Representationscited by 7
  20. 2025UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splattingcited by 6
  21. 2025EmbodiedMAE: A Unified 3D Multi-Modal Representation for Robot Manipulationcited by 5
  22. 2026EVolSplat4D: Efficient Volume-based Gaussian Splatting for 4D Urban Scene Synthesiscited by 3
  23. 2024TREND: Unsupervised 3D Representation Learning via Temporal Forecasting for LiDAR Perceptioncited by 2
  24. 2025JiSAM: Alleviate Labeling Burden and Corner Case Problems in Autonomous Driving via Minimal Real-World Datacited by 2
  25. 2025Gaussian2Scene: 3D Scene Representation Learning via Self-supervised Learning with 3D Gaussian Splattingcited by 2
  26. 2025GaussianCross: Cross-modal Self-supervised 3D Representation Learning via Gaussian Splattingcited by 2
  27. 2025Masked Clustering Prediction for Unsupervised Point Cloud Pre-trainingcited by 2
  28. 2025Weakly and Self-Supervised Class-Agnostic Motion Prediction for Autonomous Drivingcited by 2
  29. 2024Learning Shared RGB-D Fields: Unified Self-supervised Pre-training for Label-efficient LiDAR-Camera 3D Perceptioncited by 1
  30. 2024Point Cloud Unsupervised Pre-training via 3D Gaussian Splattingcited by 1
  31. 2024CLAP: Unsupervised 3D Representation Learning for Fusion 3D Perception via Curvature Sampling and Prototype Learningcited by 1
  32. 2025DynaRend: Learning 3D Dynamics via Masked Future Rendering for Robotic Manipulationcited by 1
  33. 2026PointCSP: Cross-Sample Semantic Propagation and Stability Preservation in Self-Supervised Point Cloud Learningcited by 1
  34. 2026Learning Structural Latent Points for Efficient Visual Representations in Robotic Manipulationcited by 1
  35. 2026From Extrinsic to Intrinsic: Geodesic-Guided Representation Learning for 3D Geometric Datacited by 1
  36. 2025Temporal Overlapping Prediction: A Self-supervised Pre-training Method for LiDAR Moving Object Segmentationcited by 0
  37. 2025RefComp: A Reference-guided Unified Framework for Unpaired Point Cloud Completioncited by 0
  38. 2026LASAR: Towards Spatio-temporal Reasoning with Latent Cognitive Mapcited by 0