mlsanityData to 14 Oct 2026

Point-Cloud Completion with Pretrained Text-to-image Diffusion Models

arXiv 2306.10533 ↗First posted 18 Jun 2023cs.CVcited by 37

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Point-cloud data collected in real-world applications are often incomplete. Data is typically missing due to objects being observed from partial viewpoints, which only capture a specific perspective or angle. Additionally, data can be incomplete due to occlusion and low-resolution sampling. Existing completion approaches rely on datasets of predefined objects to guide the completion of noisy and incomplete, point clouds. However, these approaches perform poorly when tested on Out-Of-Distribution (OOD) objects, that are poorly represented in the training dataset. Here we leverage recent advances in text-guided image generation, which lead to major breakthroughs in text-guided shape generation. We describe an approach called SDS-Complete that uses a pre-trained text-to-image diffusion model and leverages the text semantics of a given incomplete point cloud of an object, to obtain a complete surface representation. SDS-Complete can complete a variety of objects using test-time optimization without expensive collection of 3D information. We evaluate SDS Complete on incomplete scanned objects, captured by real-world depth sensors and LiDAR scanners. We find that it effectively reconstructs objects that are absent from common datasets, reducing Chamfer loss by 50% on average compared with current methods. Project page: https://sds-complete.github.io/

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 48 of its 61 references that are in the map. First posted on arXiv 18 Jun 2023.

Builds on

48 of 61 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
  2. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Models20 Mar
  3. 2023TEXTure: Text-Guided Texturing of 3D Shapes3 Feb
  4. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  5. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  8. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  9. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  10. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  11. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  12. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
31 more
  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  2. 2021Volume Rendering of Neural Implicit Surfacescited by 996
  3. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  4. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  5. 2020Convolutional Occupancy Networkscited by 761
  6. 2020Implicit Geometric Regularization for Learning Shapescited by 710
  7. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 588
  8. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  9. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  10. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 390
  11. 2018PCN: Point Completion Networkcited by 384
  12. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 334
  13. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 306
  14. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 288
  15. 2021PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformerscited by 286
  16. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  17. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  18. 2016Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesiscited by 219
  19. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  20. 2021SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformercited by 164
  21. 2022Diffusion-SDF: Text-to-Shape via Voxelized Diffusioncited by 128
  22. 2022ShapeFormer: Transformer-based Shape Completion via Sparse Representationcited by 106
  23. 2020Self-Supervised Learning for Domain Adaptation on Point-Cloudscited by 99
  24. 2017Hierarchical Surface Prediction for 3D Object Reconstructioncited by 89
  25. 2021Unsupervised 3D Shape Completion through GAN Inversioncited by 70
  26. 2016A Large Dataset of Object Scanscited by 64
  27. 2020Multimodal Shape Completion via Conditional Generative Adversarial Networkscited by 61
  28. 2021Neural Fields as Learnable Kernels for 3D Reconstructioncited by 58
  29. 2018Learning 3D Shape Completion under Weak Supervisioncited by 40
  30. 2022Cross-modal Learning for Image-Guided Point Cloud Shape Completioncited by 36
  31. 2017FoldingNet: Point Cloud Auto-encoder via Deep Grid Deformationcited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 18; 2025: 8; 2026: 10.

37 papers.

Sort:

By area:

  1. 2025Amodal3R: Amodal 3D Reconstruction from Occluded 2D Imagescited by 54
  2. 2024Text2LiDAR: Text-guided LiDAR Point Cloud Generation via Equirectangular Transformercited by 27
  3. 2024GALA: Generating Animatable Layered Assets from a Single Scancited by 21
  4. 2024P2P-Bridge: Diffusion Bridges for 3D Point Cloud Denoisingcited by 18
  5. 2025DecoupledGaussian: Object-Scene Decoupling for Physics-Based Interactioncited by 17
  6. 2024PCDreamer: Point Cloud Completion Through Multi-view Diffusion Priorscited by 16
  7. 2024Diffusion Models in 3D Vision: A Surveycited by 11
  8. 2024SC-Diff: 3D Shape Completion with Latent Diffusion Modelscited by 10
  9. 2024GenRC: Generative 3D Room Completion from Sparse Image Collectionscited by 9
  10. 2024Advancing 3D Point Cloud Understanding through Deep Transfer Learning: A Comprehensive Surveycited by 9
  11. 2025GenPC: Zero-shot Point Cloud Completion via 3D Generative Priorscited by 9
  12. 2026LaS-Comp: Zero-shot 3D Completion with Latent-Spatial Consistencycited by 9
25 more
  1. 2024ComPC: Completing a 3D Point Cloud with 2D Diffusion Priorscited by 6
  2. 2024Unsupervised 3D Point Cloud Completion via Multi-view Adversarial Learningcited by 6
  3. 2024Tactile DreamFusion: Exploiting Tactile Sensing for 3D Generationcited by 6
  4. 2025Diffusion Models are Open-World Affordance Learners: Leveraging Generative Priors for 3D Affordance Learningcited by 6
  5. 2024Bringing Objects to Life: training-free 4D generation from 3D objects through view consistent noisecited by 5
  6. 2024Cultural Heritage 3D Reconstruction with Diffusion Networkscited by 4
  7. 2026Points-to-3D: Structure-Aware 3D Generation with Point Cloud Priorscited by 4
  8. 2025US-X Complete: A Multi-Modal Approach to Anatomical 3D Shape Recoverycited by 3
  9. 2025Leveraging Automatic CAD Annotations for Supervised Learning in 3D Scene Understandingcited by 2
  10. 2024Enhancing Performance of Point Cloud Completion Networks with Consistency Losscited by 1
  11. 2026TouchAnything: Diffusion-Guided 3D Reconstruction from Sparse Robot Touchescited by 1
  12. 2026SplAttN: Bridging 2D and 3D with Gaussian Soft Splatting and Attention for Point Cloud Completioncited by 1
  13. 2023NeuSD: Surface Completion with Multi-View Text-to-Image Diffusioncited by 0
  14. 2024HiCAST: Highly Customized Arbitrary Style Transfer with Adapter Enhanced Diffusion Modelscited by 0
  15. 2024GPN: Generative Point-based NeRFcited by 0
  16. 2024VP-LLM: Text-Driven 3D Volume Completion with Large Language Models through Patchificationcited by 0
  17. 2024Robust Bayesian Scene Reconstruction with Retrieval-Augmented Priors for Precise Grasping and Planningcited by 0
  18. 2025SCORP: Scene-Consistent Object Refinement via Proxy Generation and Tuningcited by 0
  19. 2025Wave-Former: Through-Occlusion 3D Reconstruction via Wireless Shape Completioncited by 0
  20. 2026PQDT: Pseudo-Query Dual Transformer for Robust Point Cloud Restorationcited by 0
  21. 2026Restore3D: Breathing Life into Broken Objects with Shape and Texture Restorationcited by 0
  22. 2026MAGE: View-guided Point Cloud Completion with Efficient Modality Alignment and Adaptive Geometry Enhancementcited by 0
  23. 2026SUMI: Scalable Unified Model for 3D Point Cloud Inferencecited by 0
  24. 2026DiffSAC: Diffusion-guided Sampling for Consensus-based Robust Estimationcited by 0
  25. 2026GSComplete: Gaussian Splat Completion with 2D Diffusion Priorscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion
  2. 2024RealDiff: Real-world 3D Shape Completion using Self-Supervised Diffusion Models
  3. 2024Position-aware Guided Point Cloud Completion with CLIP Model
  4. 2024Diffusion-Occ: 3D Point Cloud Completion via Occupancy Diffusion
  5. 2023Fine-grained Text and Image Guided Point Cloud Completion with CLIP Model
  6. 2025Guided and Unguided Conditional Diffusion Mechanisms for Structured and Semantically-Aware 3D Point Cloud Generation
  7. 2024Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion
  8. 2025DiffPCN: Latent Diffusion Model Based on Multi-view Depth Images for Point Cloud Completion