mlsanityData to 14 Oct 2026

RealDiff: Real-world 3D Shape Completion using Self-Supervised Diffusion Models

arXiv 2409.10180 ↗First posted 16 Sep 2024cs.CVcited by 0

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Point cloud completion aims to recover the complete 3D shape of an object from partial observations. While approaches relying on synthetic shape priors achieved promising results in this domain, their applicability and generalizability to real-world data are still limited. To tackle this problem, we propose a self-supervised framework, namely RealDiff, that formulates point cloud completion as a conditional generation problem directly on real-world measurements. To better deal with noisy observations without resorting to training on synthetic data, we leverage additional geometric cues. Specifically, RealDiff simulates a diffusion process at the missing object parts while conditioning the generation on the partial input to address the multimodal nature of the task. We further regularize the training by matching object silhouettes and depth maps, predicted by our method, with the externally estimated ones. Experimental results show that our method consistently outperforms state-of-the-art methods in real-world point cloud completion.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 53 of its 65 references that are in the map. First posted on arXiv 16 Sep 2024.

Builds on

53 of 65 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023DiffComplete: Diffusion-based Generative 3D Shape Completion28 Jun
  2. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusion2 Dec
  3. 20223D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Models1 Dec
  4. 2022Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functions24 Nov
  5. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  6. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  7. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  8. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  10. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  11. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  12. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
35 more
  1. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  2. 2018Group Normalizationcited by 1,524
  3. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  4. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,476
  5. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  6. 2020Convolutional Occupancy Networkscited by 761
  7. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 672
  8. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  9. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  10. 2018PCN: Point Completion Networkcited by 384
  11. 2021GAN Inversion: A Surveycited by 367
  12. 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
  13. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  14. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 288
  15. 2021PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformerscited by 286
  16. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 260
  17. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  18. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matchingcited by 183
  19. 2020GRNet: Gridding Residual Network for Dense Point Cloud Completioncited by 174
  20. 2021SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformercited by 164
  21. 2020PF-Net: Point Fractal Network for 3D Point Cloud Completioncited by 146
  22. 2018Scan2CAD: Learning CAD Model Alignment in RGB-D Scanscited by 142
  23. 2020Cascaded Refinement Network for Point Cloud Completioncited by 112
  24. 2020Point Cloud Completion by Skip-attention Network with Hierarchical Foldingcited by 106
  25. 2021Variational Relational Point Completion Networkcited by 104
  26. 2020PMP-Net: Point Cloud Completion by Learning Multi-step Point Moving Pathscited by 97
  27. 2020Detail Preserved Point Cloud Completion via Separated Feature Aggregationcited by 70
  28. 2021Unsupervised 3D Shape Completion through GAN Inversioncited by 70
  29. 2023AdaPoinTr: Diverse Point Cloud Completion with Adaptive Geometry-Aware Transformerscited by 67
  30. 2019Unpaired Point Cloud Completion on Real Scans using Adversarial Trainingcited by 66
  31. 2016A Large Dataset of Object Scanscited by 64
  32. 2021Cycle4Completion: Unpaired Point Cloud Completion using Cycle Transformation with Missing Region Codingcited by 50
  33. 2020Weakly-supervised 3D Shape Completion in the Wildcited by 33
  34. 2022LAKe-Net: Topology-Aware Point Cloud Completion by Localizing Aligned Keypointscited by 27
  35. 2023P2C: Self-Supervised Point Cloud Completion from Single Partial Cloudscited by 20

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024SC-Diff: 3D Shape Completion with Latent Diffusion Models
  2. 2021A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion
  3. 2024PCDreamer: Point Cloud Completion Through Multi-view Diffusion Priors
  4. 2025Repurposing 2D Diffusion Models for 3D Shape Completion
  5. 2023Point-Cloud Completion with Pretrained Text-to-image Diffusion Models
  6. 2025Self-Supervised Point Cloud Completion based on Multi-View Augmentations of Single Partial Point Cloud
  7. 2024ComPC: Completing a 3D Point Cloud with 2D Diffusion Priors
  8. 2024Digging into Intrinsic Contextual Information for High-fidelity 3D Point Cloud Completion