mlsanityData to 7 Oct 2026

Cross-modal Learning for Image-Guided Point Cloud Shape Completion

arXiv 2209.09552 ↗First posted 20 Sep 2022cs.CV eess.IVcited by 36

Not on the map. Why?

Abstract (arXiv)

In this paper we explore the recent topic of point cloud completion, guided by an auxiliary image. We show how it is possible to effectively combine the information from the two modalities in a localized latent space, thus avoiding the need for complex point cloud reconstruction methods from single views used by the state-of-the-art. We also investigate a novel weakly-supervised setting where the auxiliary image provides a supervisory signal to the training process by using a differentiable renderer on the completed point cloud to measure fidelity in the image space. Experiments show significant improvements over state-of-the-art supervised methods for both unimodal and multimodal completion. We also show the effectiveness of the weakly-supervised approach which outperforms a number of supervised methods and is competitive with the latest supervised models only exploiting point cloud information.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 30 of its 46 references that are in the map. First posted on arXiv 20 Sep 2022.

Builds on

30 of 46 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  6. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  7. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  8. 2019Self-Attention Graph Poolingcited by 421
  9. 2018PCN: Point Completion Networkcited by 382
  10. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  11. 2021PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformerscited by 284
  12. 2016Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesiscited by 219
18 more
  1. 20183D-SIS: 3D Semantic Instance Segmentation of RGB-D Scanscited by 216
  2. 2020GRNet: Gridding Residual Network for Dense Point Cloud Completioncited by 174
  3. 2021SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformercited by 162
  4. 2020PF-Net: Point Fractal Network for 3D Point Cloud Completioncited by 146
  5. 2019Morphing and Sampling Network for Dense Point Cloud Completioncited by 132
  6. 2020Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Reviewcited by 114
  7. 2021Variational Relational Point Completion Networkcited by 103
  8. 2018Unsupervised Learning of Shape and Pose with Differentiable Point Cloudscited by 89
  9. 2017High-Resolution Shape Completion Using Deep Neural Networks for Global Structure and Local Geometry Inferencecited by 87
  10. 2016Shape Completion Enabled Robotic Graspingcited by 79
  11. 2021View-Guided Point Cloud Completioncited by 46
  12. 2021Style-based Point Generator with Adversarial Rendering for Point Cloud Completioncited by 45
  13. 2022Comprehensive Review of Deep Learning-Based 3D Point Cloud Completion Processing and Analysiscited by 44
  14. 2020Cascaded Refinement Network for Point Cloud Completion with Self-supervisioncited by 20
  15. 2021Denoise and Contrast for Category Agnostic Shape Completioncited by 17
  16. 2021Self-Supervised Point Cloud Completion via Inpaintingcited by 16
  17. 2020Geometric Correspondence Fields: Learned Differentiable Rendering for 3D Pose Refinement in the Wildcited by 10
  18. 2021Attention-based Transformation from Latent Features to Point Cloudscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 10; 2025: 12; 2026: 9.

36 papers.

Sort:

By area:

  1. 2023Grounding 3D Object Affordance from 2D Interactions in Imagescited by 62
  2. 2023Point-Cloud Completion with Pretrained Text-to-image Diffusion Modelscited by 37
  3. 2023SVDFormer: Complementing Point Cloud via Self-view Augmentation and Self-structure Dual-generatorcited by 34
  4. 2024GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Groundingcited by 26
  5. 2024Learning 2D Invariant Affordance Knowledge for 3D Affordance Groundingcited by 21
  6. 2025InteractVLM: 3D Interaction Reasoning from 2D Foundational Modelscited by 21
  7. 2025SuperPC: A Single Diffusion Model for Point Cloud Completion, Upsampling, Denoising, and Colorizationcited by 20
  8. 2024GEAL: Generalizable 3D Affordance Learning with Cross-Modal Consistencycited by 17
  9. 2024PCDreamer: Point Cloud Completion Through Multi-view Diffusion Priorscited by 16
  10. 2024Explicitly Guided Information Interaction Network for Cross-modal Point Cloud Completioncited by 9
  11. 2025PointSea: Point Cloud Completion via Self-structure Augmentationcited by 9
  12. 2022Leveraging Single-View Images for Unsupervised 3D Point Cloud Completioncited by 8
24 more
  1. 2024Unsupervised 3D Point Cloud Completion via Multi-view Adversarial Learningcited by 6
  2. 2023Fine-grained Text and Image Guided Point Cloud Completion with CLIP Modelcited by 5
  3. 2024DMF-Net: Image-Guided Point Cloud Completion with Dual-Channel Modality Fusion and Shape-Aware Upsampling Transformercited by 5
  4. 2025Diffusion Models are Open-World Affordance Learners: Leveraging Generative Priors for 3D Affordance Learningcited by 5
  5. 2025DiffPCN: Latent Diffusion Model Based on Multi-view Depth Images for Point Cloud Completioncited by 5
  6. 2024Deep Learning for 3D Point Cloud Enhancement: A Surveycited by 4
  7. 2025MambaTron: Efficient Cross-Modal Point Cloud Enhancement using Aggregate Selective State Space Modelingcited by 2
  8. 2025Unlocking 3D Affordance Segmentation with 2D Semantic Knowledgecited by 2
  9. 2026MGPC: Multimodal Network for Generalizable Point Cloud Completion With Modality Dropout and Progressive Decodingcited by 2
  10. 2024DuInNet: Dual-Modality Feature Interaction for Point Cloud Completioncited by 1
  11. 2025Benefit from Reference: Retrieval-Augmented Cross-modal Point Cloud Completioncited by 1
  12. 2025Object Affordance Recognition and Grounding via Multi-scale Cross-modal Representation Learningcited by 1
  13. 2025HeartFormer: Semantic-Aware Dual-Structure Transformers for 3D Four-Chamber Cardiac Point Cloud Reconstructioncited by 1
  14. 2026VAGNet: Grounding 3D Affordance from Human-Object Interactions in Videoscited by 1
  15. 2026HAMMER: Harnessing MLLM via Cross-Modal Integration for Intention-Driven 3D Affordance Groundingcited by 1
  16. 2026SplAttN: Bridging 2D and 3D with Gaussian Soft Splatting and Attention for Point Cloud Completioncited by 1
  17. 2026I2PRef: Image-Driven Point Completion with Iterative Refinementcited by 1
  18. 2024MASSTAR: A Multi-Modal and Large-Scale Scene Dataset with a Versatile Toolchain for Surface Prediction and Completioncited by 0
  19. 2025A Strong View-Free Baseline Approach for Single-View Image Guided Point Cloud Completioncited by 0
  20. 2025A Multi-View High-Resolution Foot-Ankle Complex Point Cloud Dataset During Gait for Occlusion-Robust 3D Completioncited by 0
  21. 2026Part-Aware Open-Vocabulary 3D Affordance Grounding via Prototypical Semantic and Geometric Alignmentcited by 0
  22. 2026GROW2: Grounding Which and Where for Robot Tool Usecited by 0
  23. 2026MAGE: View-guided Point Cloud Completion with Efficient Modality Alignment and Adaptive Geometry Enhancementcited by 0
  24. 2026Denoising-Aware Temporal Point Cloud Completion for 3D Crop Architecture Recovery and Phenotypic Trait Extractioncited by 0