mlsanityData to 7 Oct 2026

PC2: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction

arXiv 2302.10668 ↗First posted 21 Feb 2023cs.CV cs.AI cs.LGcited by 84

Not on the map. Why?

Cited by 29 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Reconstructing the 3D shape of an object from a single RGB image is a long-standing and highly challenging problem in computer vision. In this paper, we propose a novel method for single-image 3D reconstruction which generates a sparse point cloud via a conditional denoising diffusion process. Our method takes as input a single RGB image along with its camera pose and gradually denoises a set of 3D points, whose positions are initially sampled randomly from a three-dimensional Gaussian distribution, into the shape of an object. The key to our method is a geometrically-consistent conditioning process which we call projection conditioning: at each step in the diffusion process, we project local image features onto the partially-denoised point cloud from the given camera pose. This projection conditioning process enables us to generate high-resolution sparse geometries that are well-aligned with the input image, and can additionally be used to predict point colors after shape reconstruction. Moreover, due to the probabilistic nature of the diffusion process, our method is naturally capable of generating multiple different shapes consistent with a single input image. In contrast to prior work, our approach not only performs well on synthetic benchmarks, but also gives large qualitative improvements on complex real-world data.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). Builds on 46 of its 55 references that are in the map. First posted on arXiv 21 Feb 2023.

Builds on

46 of 55 references.

References · 46

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2020Denoising Diffusion Implicit Modelscited by 9,655
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  9. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  11. 2017Mask R-CNNcited by 7,259
  12. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
34 more
  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  5. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  6. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  7. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  8. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  9. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  10. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  11. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 734
  12. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 695
  13. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  14. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  15. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  16. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  17. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  18. 2019Point-Voxel CNN for Efficient 3D Deep Learningcited by 343
  19. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 259
  20. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  21. 2017Learning a Multi-View Stereo Machinecited by 194
  22. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  23. 2022Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Qualitycited by 181
  24. 2019Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Imagescited by 164
  25. 2021A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completioncited by 100
  26. 2021ShaRF: Shape-conditioned Radiance Fields from a Single Viewcited by 92
  27. 2020Pix2Vox++: Multi-scale Context-aware 3D Object Reconstruction from Single and Multiple Imagescited by 79
  28. 2022ViewFormer: NeRF-free Neural Rendering from Few Images Using Transformerscited by 70
  29. 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
  30. 2019Single-Image Piece-wise Planar 3D Reconstruction via Associative Embeddingcited by 56
  31. 2018Learning View Priors for Single-view 3D Reconstructioncited by 35
  32. 2019Photometric Mesh Optimization for Video-Aligned 3D Object Reconstructioncited by 28
  33. 2019Few-Shot Generalization for Single-Image 3D Reconstruction via Priorscited by 25
  34. 2021LegoFormer: Transformers for Block-by-Block Multi-view 3D Reconstructioncited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 17; 2024: 30; 2025: 25; 2026: 12.

84 papers.

Sort:

By area:

  1. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  2. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  3. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  4. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  5. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 257
  6. 2023PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustmentcited by 146
  7. 2024latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstructioncited by 114
  8. 2023Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Modelscited by 79
  9. 2025SparseFlex: High-Resolution and Arbitrary-Topology 3D Shape Modelingcited by 76
  10. 2023Free3D: Consistent Novel View Synthesis without 3D Representationcited by 72
  11. 2024Towards Realistic Scene Generation with LiDAR Diffusion Modelscited by 67
  12. 2024PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Modelscited by 60
38 more
  1. 2025Amodal3R: Amodal 3D Reconstruction from Occluded 2D Imagescited by 52
  2. 2024GVGEN: Text-to-3D Generation with Volumetric Representationcited by 51
  3. 2024Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learningcited by 43
  4. 2023ShapeGPT: 3D Shape Generation with A Unified Multi-modal Language Modelcited by 38
  5. 2023Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generationcited by 37
  6. 2024GeoLRM: Geometry-Aware Large Reconstruction Model for High-Quality 3D Gaussian Generationcited by 35
  7. 2023A Conditional Denoising Diffusion Probabilistic Model for Point Cloud Upsamplingcited by 28
  8. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
  9. 2024Query2CAD: Generating CAD models using natural language queriescited by 28
  10. 2023HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3Dcited by 27
  11. 2024Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectivescited by 27
  12. 2024DRiVE: Diffusion-based Rigging Empowers Generation of Versatile and Expressive Characterscited by 24
  13. 2024Neuro-3D: Towards 3D Visual Decoding from EEG Signalscited by 23
  14. 2024InterTrack: Tracking Human Object Interaction without Object Templatescited by 22
  15. 2025SuperPC: A Single Diffusion Model for Point Cloud Completion, Upsampling, Denoising, and Colorizationcited by 20
  16. 2024Bayesian Diffusion Models for 3D Shape Reconstructioncited by 19
  17. 2023InstructP2P: Learning to Edit 3D Point Clouds with Text Instructionscited by 14
  18. 2024EMIE-MAP: Large-Scale Road Surface Reconstruction Based on Explicit Mesh and Implicit Encodingcited by 13
  19. 2024Lightplane: Highly-Scalable Components for Neural 3D Fieldscited by 13
  20. 2025Single View Garment Reconstruction Using Diffusion Mapping Via Pattern Coordinatescited by 12
  21. 2025HORT: Monocular Hand-held Objects Reconstruction with Transformerscited by 11
  22. 2023CCD-3DR: Consistent Conditioning in Diffusion for Single-Image 3D Reconstructioncited by 10
  23. 2024Large Point-to-Gaussian Model for Image-to-3D Generationcited by 10
  24. 2024Boosting 3D Object Generation through PBR Materialscited by 10
  25. 2025Not All Frame Features Are Equal: Video-to-4D Generation via Decoupling Dynamic-Static Featurescited by 10
  26. 2024SDFit: 3D Object Pose and Shape by Fitting a Morphable SDF to a Single Imagecited by 9
  27. 2024Reconstruction of Manipulated Garment with Guided Deformation Priorcited by 8
  28. 2024Gaussian Scenes: Pose-Free Sparse-View Scene Reconstruction using Depth-Enhanced Diffusion Priorscited by 8
  29. 2024Coherent 3D Scene Diffusion From a Single RGB Imagecited by 8
  30. 2024EC-Diffuser: Multi-Object Manipulation via Entity-Centric Behavior Generationcited by 8
  31. 2024Sp2360: Sparse-view 360 Scene Reconstruction using Cascaded 2D Diffusion Priorscited by 7
  32. 2024RGB2Point: 3D Point Cloud Generation from Single RGB Imagescited by 7
  33. 2025Digital Twin Generation from Visual Data: A Surveycited by 7
  34. 2023BuilDiff: 3D Building Shape Generation using Single-Image Conditional Point Cloud Diffusion Modelscited by 6
  35. 2024DiffuBox: Refining 3D Object Detection with Point Diffusioncited by 6
  36. 2024ScalingGaussian: Enhancing 3D Content Creation with Generative Gaussian Splattingcited by 6
  37. 2025Is Single-View Mesh Reconstruction Ready for Robotics?cited by 6
  38. 2023Coordinate Quantized Neural Implicit Representations for Multi-view Reconstructioncited by 5

50 most cited of 84 All 84 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Baking Gaussian Splatting into Diffusion Denoiser for Fast and Scalable Single-stage Image-to-3D Generation and Reconstruction
  2. 2023GECCO: Geometrically-Conditioned Point Diffusion Models
  3. 20233D Colored Shape Reconstruction from a Single RGB Image through Diffusion
  4. 2024Point Cloud Resampling with Learnable Heat Diffusion
  5. 2024Few-shot point cloud reconstruction and denoising via learned Guassian splats renderings and fine-tuned diffusion features
  6. 2025Consistency Diffusion Models for Single-Image 3D Reconstruction with Priors
  7. 2025SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Images
  8. 2023PCRDiffusion: Diffusion Probabilistic Models for Point Cloud Registration