mlsanityData to 7 Oct 2026

SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scans

arXiv 1912.00036 ↗First posted 29 Nov 2019cs.CVcited by 77

Not on the map. Why?

Abstract (arXiv)

We present a novel approach that converts partial and noisy RGB-D scans into high-quality 3D scene reconstructions by inferring unobserved scene geometry. Our approach is fully self-supervised and can hence be trained solely on real-world, incomplete scans. To achieve self-supervision, we remove frames from a given (incomplete) 3D scan in order to make it even more incomplete; self-supervision is then formulated by correlating the two levels of partialness of the same scan while masking out regions that have never been observed. Through generalization across a large training set, we can then predict 3D scene completion without ever seeing any 3D scan of entirely complete geometry. Combined with a new 3D sparse generative neural network architecture, our method is able to predict highly-detailed surfaces in a coarse-to-fine hierarchical fashion, generating 3D scenes at 2cm resolution, more than twice the resolution of existing state-of-the-art methods as well as outperforming them by a significant margin in reconstruction quality.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 17; 2023: 17; 2024: 14; 2025: 8; 2026: 2.

77 papers.

Sort:

By area:

  1. 2021iMAP: Implicit Mapping and Positioning in Real-Timecited by 476
  2. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 449
  3. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  4. 2021Neural RGB-D Surface Reconstructioncited by 312
  5. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
  6. 2021Dense Depth Priors for Neural Radiance Fields from Sparse Input Viewscited by 301
  7. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 254
  8. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  9. 2021Shape As Points: A Differentiable Poisson Solvercited by 186
  10. 20214DComplete: Non-Rigid Motion Estimation Beyond the Observable Surfacecited by 123
  11. 2021TransformerFusion: Monocular RGB Scene Reconstruction using Transformerscited by 107
  12. 2022POCO: Point Convolution for Surface Reconstructioncited by 91
38 more
  1. 2023OccDepth: A Depth-Aware Method for 3D Semantic Scene Completioncited by 85
  2. 20213D Semantic Scene Completion: a Surveycited by 74
  3. 2021Panoptic 3D Scene Reconstruction From a Single RGB Imagecited by 70
  4. 2022Texturify: Generating Textures on 3D Shape Surfacescited by 65
  5. 2021StyleMesh: Style Transfer for Indoor 3D Scene Reconstructionscited by 58
  6. 2024InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Priorcited by 55
  7. 2023RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texturecited by 53
  8. 2024LT3SD: Latent Trees for 3D Scene Diffusioncited by 53
  9. 2023PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awarenesscited by 42
  10. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 41
  11. 2024GScream: Learning 3D Geometry and Feature Consistent Gaussian Splatting for Object Removalcited by 39
  12. 2023DiffComplete: Diffusion-based Generative 3D Shape Completioncited by 35
  13. 2021RetrievalFuse: Neural 3D Scene Reconstruction with a Databasecited by 28
  14. 2021Indoor Scene Generation from a Collection of Semantic-Segmented Depth Imagescited by 27
  15. 2024DepthLab: From Partial to Completecited by 27
  16. 2021Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Mappingcited by 25
  17. 2024PhysPart: Physically Plausible Part Completion for Interactable Objectscited by 24
  18. 2022ALTO: Alternating Latent Topologies for Implicit 3D Reconstructioncited by 21
  19. 2022Point Scene Understanding via Disentangled Instance Mesh Reconstructioncited by 20
  20. 2023SGRec3D: Self-Supervised 3D Scene Graph Learning via Object-Level Scene Reconstructioncited by 20
  21. 2026ShapeR: Robust Conditional 3D Shape Generation from Casual Capturescited by 19
  22. 2023FineRecon: Depth-aware Feed-forward Network for Detailed 3D Reconstructioncited by 16
  23. 2022Multi-sensor large-scale dataset for multi-view 3D reconstructioncited by 14
  24. 2023MV-DeepSDF: Implicit Modeling with Multi-Sweep Point Clouds for 3D Vehicle Reconstruction in Autonomous Drivingcited by 11
  25. 2023Learning Neural Implicit through Volume Rendering with Attentive Depth Fusion Priorscited by 11
  26. 2025QuickSplat: Fast 3D Surface Reconstruction via Learned Gaussian Initializationcited by 11
  27. 2024SC-Diff: 3D Shape Completion with Latent Diffusion Modelscited by 10
  28. 2022CP3: Unifying Point Cloud Completion by Pretrain-Prompt-Predict Paradigmcited by 9
  29. 2023Clutter Detection and Removal in 3D Scenes with View-Consistent Inpaintingcited by 9
  30. 2024Zero-Shot Multi-Object Scene Completioncited by 9
  31. 2021Unsupervised Shape Completion via Deep Prior in the Neural Tangent Kernel Perspectivecited by 8
  32. 2022Deep Surface Reconstruction from Point Clouds with Visibility Informationcited by 8
  33. 2022Probabilistic Implicit Scene Completioncited by 8
  34. 2022Semi-signed prioritized neural fitting for surface reconstruction from unoriented point cloudscited by 8
  35. 2021What's Behind the Couch? Directed Ray Distance Functions (DRDF) for 3D Scene Reconstructioncited by 7
  36. 2023SSR-2D: Semantic 3D Scene Reconstruction from 2D Imagescited by 7
  37. 2025Towards Learning to Complete Anything in Lidarcited by 7
  38. 2022Neural Part Priors: Learning to Optimize Part-Based Object Completion in RGB-D Scanscited by 6

50 most cited of 77 All 77 (JSON)