mlsanityData to 7 Oct 2026

RGB2Point: 3D Point Cloud Generation from Single RGB Images

arXiv 2407.14979 ↗First posted 20 Jul 2024cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

We introduce RGB2Point, an unposed single-view RGB image to a 3D point cloud generation based on Transformer. RGB2Point takes an input image of an object and generates a dense 3D point cloud. Contrary to prior works based on CNN layers and diffusion denoising approaches, we use pre-trained Transformer layers that are fast and generate high-quality point clouds with consistent quality over available categories. Our generated point clouds demonstrate high quality on a real-world dataset, as evidenced by improved Chamfer distance (51.15%) and Earth Mover's distance (45.96%) metrics compared to the current state-of-the-art. Additionally, our approach shows a better quality on a synthetic dataset, achieving better Chamfer distance (39.26%), Earth Mover's distance (26.95%), and F-score (47.16%). Moreover, our method produces 63.1% more consistent high-quality results across various object categories compared to prior works. Furthermore, RGB2Point is computationally efficient, requiring only 2.3GB of VRAM to reconstruct a 3D point cloud from a single RGB image, and our implementation generates the results 15,133x faster than a SOTA diffusion-based model.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 56 of its 65 references that are in the map. First posted on arXiv 20 Jul 2024.

Builds on

56 of 65 references.

References · 56

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  8. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  9. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  10. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  11. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  12. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
38 more
  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  2. 2020End-to-End Object Detection with Transformerscited by 7,384
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  7. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  9. 2021Vision Transformers for Dense Predictioncited by 1,789
  10. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  11. 2020Point Transformercited by 1,462
  12. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  13. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  14. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 734
  15. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  16. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  17. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  18. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  19. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  20. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  21. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
  22. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  23. 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
  24. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  25. 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 227
  26. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  27. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 209
  28. 2021You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detectioncited by 170
  29. 2019Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Imagescited by 164
  30. 2022MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformercited by 136
  31. 2022SegViT: Semantic Segmentation with Plain Vision Transformerscited by 105
  32. 2021Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognitioncited by 103
  33. 2021A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completioncited by 100
  34. 2021Multi-Scale High-Resolution Vision Transformer for Semantic Segmentationcited by 92
  35. 2018Unsupervised Learning of Shape and Pose with Differentiable Point Cloudscited by 89
  36. 2023PC2: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstructioncited by 84
  37. 2020Pix2Vox++: Multi-scale Context-aware 3D Object Reconstruction from Single and Multiple Imagescited by 79
  38. 2023Free3D: Consistent Novel View Synthesis without 3D Representationcited by 72

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025Uncertainty-o: One Model-agnostic Framework for Unveiling Uncertainty in Large Multimodal Modelscited by 4
  2. 2025RadarGen: Automotive Radar Point Cloud Generation from Camerascited by 2
  3. 2025Adaptive 3D Reconstruction via Diffusion Priors and Forward Curvature-Matching Likelihood Updatescited by 1
  4. 2026I2PRef: Image-Driven Point Completion with Iterative Refinementcited by 1
  5. 2025RL-AD-Net: Reinforcement Learning Guided Adaptive Displacement in Latent Space for Refined Point Cloud Completioncited by 0
  6. 2026Point-MF: Stabilizing One-Step Mean Flows for Single-Image Point Cloud Reconstructioncited by 0
  7. 2026PrintAnything: Learning an Intermediate Representation for 3D printing G-code Generationcited by 0