mlsanityData to 7 Oct 2026

UNeR3D: Versatile and Scalable 3D RGB Point Cloud Generation from 2D Images in Unsupervised Reconstruction

arXiv 2312.06706 ↗First posted 10 Dec 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

In the realm of 3D reconstruction from 2D images, a persisting challenge is to achieve high-precision reconstructions devoid of 3D Ground Truth data reliance. We present UNeR3D, a pioneering unsupervised methodology that sets a new standard for generating detailed 3D reconstructions solely from 2D views. Our model significantly cuts down the training costs tied to supervised approaches and introduces RGB coloration to 3D point clouds, enriching the visual experience. Employing an inverse distance weighting technique for color rendering, UNeR3D ensures seamless color transitions, enhancing visual fidelity. Our model's flexible architecture supports training with any number of views, and uniquely, it is not constrained by the number of views used during training when performing reconstructions. It can infer with an arbitrary count of views during inference, offering unparalleled versatility. Additionally, the model's continuous spatial input domain allows the generation of point clouds at any desired resolution, empowering the creation of high-resolution 3D RGB point clouds. We solidify the reconstruction process with a novel multi-view geometric loss and color loss, demonstrating that our model excels with single-view inputs and beyond, thus reshaping the paradigm of unsupervised learning in 3D vision. Our contributions signal a substantial leap forward in 3D vision, offering new horizons for content creation across diverse applications. Code is available at https://github.com/HongbinLin3589/UNeR3D.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 62 references that are in the map. First posted on arXiv 10 Dec 2023.

Builds on

40 of 62 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  6. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  7. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  8. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  9. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  10. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  11. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,182
  12. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
28 more
  1. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  2. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  3. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  4. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  5. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  6. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  7. 2017Neural 3D Mesh Renderercited by 437
  8. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 340
  9. 2021PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformerscited by 284
  10. 2016Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesiscited by 219
  11. 2023Text-To-4D Dynamic Scene Generationcited by 204
  12. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  13. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  14. 2019Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Imagescited by 164
  15. 2022HF-NeuS: Improved Surface Reconstruction Using High-Frequency Detailscited by 120
  16. 2016Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervisioncited by 106
  17. 2019Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Eracited by 87
  18. 2023Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflectioncited by 68
  19. 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
  20. 2023Parameter is Not All You Need: Starting from Non-Parametric Networks for 3D Point Cloud Analysiscited by 34
  21. 2019Dense 3D Point Cloud Reconstruction Using a Deep Pyramid Networkcited by 33
  22. 2023NeAT: Learning Neural Implicit Surfaces with Arbitrary Topologies from Multi-view Imagescited by 32
  23. 2017Image2Mesh: A Learning Framework for Single Image 3D Reconstructioncited by 29
  24. 2023Robotic Perception of Transparent Objects: A Reviewcited by 28
  25. 2017Shape Inpainting using 3D Generative Adversarial Network and Recurrent Convolutional Networkscited by 23
  26. 2018MVPNet: Multi-View Point Regression Networks for 3D Object Reconstruction from A Single Imagecited by 17
  27. 2018CAPNet: Continuous Approximation Projection For 3D Point Cloud Reconstruction Using 2D Supervisioncited by 12
  28. 2023Adv3D: Generating 3D Adversarial Examples for 3D Object Detection in Driving Scenarios with NeRFcited by 10

Built on by

Not cited yet.