# A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion

arXiv: https://arxiv.org/abs/2112.03530 · HTML: https://mlsanity.com/paper/2112.03530/

## Key facts

- Cited by 100 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 27 of its 34 references that are in the map.
- First posted on arXiv 7 Dec 2021.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [3D Shape Generation and Completion through Point-Voxel Diffusion](https://mlsanity.com/paper/2104.03670/) (2021)
- [PCN: Point Completion Network](https://mlsanity.com/paper/1808.00671/) (2018)
- [Point-Voxel CNN for Efficient 3D Deep Learning](https://mlsanity.com/paper/1907.03739/) (2019)
- [PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers](https://mlsanity.com/paper/2108.08839/) (2021)
- [What Do Single-view 3D Reconstruction Networks Learn?](https://mlsanity.com/paper/1905.03678/) (2019)
- [GRNet: Gridding Residual Network for Dense Point Cloud Completion](https://mlsanity.com/paper/2006.03761/) (2020)
- [On Fast Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2106.00132/) (2021)
- [Searching for Activation Functions](https://mlsanity.com/paper/1710.05941/) (2017)
- [Morphing and Sampling Network for Dense Point Cloud Completion](https://mlsanity.com/paper/1912.00280/) (2019)
- [Cascaded Refinement Network for Point Cloud Completion](https://mlsanity.com/paper/2004.03327/) (2020)
- [Variational Relational Point Completion Network](https://mlsanity.com/paper/2104.10154/) (2021)
- [PMP-Net: Point Cloud Completion by Learning Multi-step Point Moving Paths](https://mlsanity.com/paper/2012.03408/) (2020)
- [Detail Preserved Point Cloud Completion via Separated Feature Aggregation](https://mlsanity.com/paper/2007.02374/) (2020)
- [Unsupervised 3D Shape Completion through GAN Inversion](https://mlsanity.com/paper/2104.13366/) (2021)
- [ASFM-Net: Asymmetrical Siamese Feature Matching Network for Point Completion](https://mlsanity.com/paper/2104.09587/) (2021)

## Built on by (most cited first, 30 of 100)

- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [A Survey on Generative Diffusion Model](https://mlsanity.com/paper/2209.02646/) (2022)
- [PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustment](https://mlsanity.com/paper/2306.15667/) (2023)
- [Solving 3D Inverse Problems using Pre-trained 2D Diffusion Models](https://mlsanity.com/paper/2211.10655/) (2022)
- [Locally Attentional SDF Diffusion for Controllable 3D Shape Generation](https://mlsanity.com/paper/2305.04461/) (2023)
- [GENIE: Higher-Order Denoising Diffusion Solvers](https://mlsanity.com/paper/2210.05475/) (2022)
- [TFG: Unified Training-Free Guidance for Diffusion Models](https://mlsanity.com/paper/2409.15761/) (2024)
- [Accelerating Diffusion Models via Early Stop of the Diffusion Process](https://mlsanity.com/paper/2205.12524/) (2022)
- [Exploring Chemical Space with Score-based Out-of-distribution Generation](https://mlsanity.com/paper/2206.07632/) (2022)
- [PC2: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction](https://mlsanity.com/paper/2302.10668/) (2023)
- [HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoising](https://mlsanity.com/paper/2211.13287/) (2022)
- [PriSTI: A Conditional Diffusion Framework for Spatiotemporal Imputation](https://mlsanity.com/paper/2302.09746/) (2023)
- [On the Design Fundamentals of Diffusion Models: A Survey](https://mlsanity.com/paper/2306.04542/) (2023)
- [Fast Point Cloud Generation with Straight Flows](https://mlsanity.com/paper/2212.01747/) (2022)
- [Scalable Diffusion for Materials Generation](https://mlsanity.com/paper/2311.09235/) (2023)
- [Face Animation with an Attribute-Guided Diffusion Model](https://mlsanity.com/paper/2304.03199/) (2023)
- [Comprehensive Review of Deep Learning-Based 3D Point Cloud Completion Processing and Analysis](https://mlsanity.com/paper/2203.03311/) (2022)
- [RF-Diffusion: Radio Signal Generation via Time-Frequency Diffusion](https://mlsanity.com/paper/2404.09140/) (2024)
- [Controllable Mesh Generation Through Sparse Latent Point Diffusion Models](https://mlsanity.com/paper/2303.07938/) (2023)
- [Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion](https://mlsanity.com/paper/2403.13470/) (2024)
- [DiffFacto: Controllable Part-Based 3D Point Cloud Generation with Cross Diffusion](https://mlsanity.com/paper/2305.01921/) (2023)
- [Human Joint Kinematics Diffusion-Refinement for Stochastic Motion Prediction](https://mlsanity.com/paper/2210.05976/) (2022)
- [A Conditional Denoising Diffusion Probabilistic Model for Point Cloud Upsampling](https://mlsanity.com/paper/2312.02719/) (2023)
- [GECCO: Geometrically-Conditioned Point Diffusion Models](https://mlsanity.com/paper/2303.05916/) (2023)
- [Advancements in Point Cloud Data Augmentation for Deep Learning: A Survey](https://mlsanity.com/paper/2308.12113/) (2023)
- [Distribution Shift Inversion for Out-of-Distribution Prediction](https://mlsanity.com/paper/2306.08328/) (2023)
- [Add and Thin: Diffusion for Temporal Point Processes](https://mlsanity.com/paper/2311.01139/) (2023)
- [Uncertainty-aware Surrogate Models for Airfoil Flow Simulations with Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2312.05320/) (2023)

## Abstract (arXiv)

3D point cloud is an important 3D representation for capturing real world 3D objects. However, real-scanned 3D point clouds are often incomplete, and it is important to recover complete point clouds for downstream applications. Most existing point cloud completion methods use Chamfer Distance (CD) loss for training. The CD loss estimates correspondences between two point clouds by searching nearest neighbors, which does not capture the overall point density distribution on the generated shape, and therefore likely leads to non-uniform point cloud generation. To tackle this problem, we propose a novel Point Diffusion-Refinement (PDR) paradigm for point cloud completion. PDR consists of a Conditional Generation Network (CGNet) and a ReFinement Network (RFNet). The CGNet uses a conditional generative model called the denoising diffusion probabilistic model (DDPM) to generate a coarse completion conditioned on the partial observation. DDPM establishes a one-to-one pointwise mapping between the generated point cloud and the uniform ground truth, and then optimizes the mean squared error loss to realize uniform generation. The RFNet refines the coarse output of the CGNet and further improves quality of the completed point cloud. Furthermore, we develop a novel dual-path architecture for both networks. The architecture can (1) effectively and efficiently extract multi-level features from partially observed point clouds to guide completion, and (2) accurately manipulate spatial locations of 3D points to obtain smooth surfaces and sharp details. Extensive experimental results on various benchmark datasets show that our PDR paradigm outperforms previous state-of-the-art methods for point cloud completion. Remarkably, with the help of the RFNet, we can accelerate the iterative generation process of the DDPM by up to 50 times without much performance drop.