mlsanityData to 7 Oct 2026

Point Scene Understanding via Disentangled Instance Mesh Reconstruction

arXiv 2203.16832 ↗First posted 31 Mar 2022cs.CVcited by 20

Not on the map. Why?

Abstract (arXiv)

Semantic scene reconstruction from point cloud is an essential and challenging task for 3D scene understanding. This task requires not only to recognize each instance in the scene, but also to recover their geometries based on the partial observed point cloud. Existing methods usually attempt to directly predict occupancy values of the complete object based on incomplete point cloud proposals from a detection-based backbone. However, this framework always fails to reconstruct high fidelity mesh due to the obstruction of various detected false positive object proposals and the ambiguity of incomplete point observations for learning occupancy values of complete objects. To circumvent the hurdle, we propose a Disentangled Instance Mesh Reconstruction (DIMR) framework for effective point scene understanding. A segmentation-based backbone is applied to reduce false positive object proposals, which further benefits our exploration on the relationship between recognition and reconstruction. Based on the accurate proposals, we leverage a mesh-aware latent code space to disentangle the processes of shape completion and mesh generation, relieving the ambiguity caused by the incomplete point observations. Furthermore, with access to the CAD model pool at test time, our model can also be used to improve the reconstruction quality by performing mesh retrieval without extra training. We thoroughly evaluate the reconstructed mesh quality with multiple metrics, and demonstrate the superiority of our method on the challenging ScanNet dataset. Code is available at urlhttps://github.com/ashawkey/dimr.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 64 of its 74 references that are in the map. First posted on arXiv 31 Mar 2022.

Builds on

64 of 74 references.

References · 64

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  6. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  7. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
  8. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  9. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  10. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  11. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 422
  12. 2018PCN: Point Completion Networkcited by 382
38 more
  1. 2019Point-Voxel CNN for Efficient 3D Deep Learningcited by 343
  2. 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
  3. 2020PointGroup: Dual-Set Point Grouping for 3D Instance Segmentationcited by 318
  4. 2017Submanifold Sparse Convolutional Networkscited by 223
  5. 2016Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesiscited by 219
  6. 20183D-SIS: 3D Semantic Instance Segmentation of RGB-D Scanscited by 216
  7. 2021Group-Free 3D Object Detection via Transformerscited by 215
  8. 2019BSP-Net: Generating Compact Meshes via Binary Space Partitioningcited by 205
  9. 2020OccuSeg: Occupancy-aware 3D Instance Segmentationcited by 177
  10. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  11. 2017SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentationcited by 168
  12. 2019Learning Object Bounding Boxes for 3D Instance Segmentation on Point Cloudscited by 156
  13. 2018Scan2CAD: Learning CAD Model Alignment in RGB-D Scanscited by 142
  14. 20203D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentationcited by 140
  15. 2021Hierarchical Aggregation for 3D Instance Segmentationcited by 126
  16. 20203D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Priorcited by 124
  17. 2019PQ-NET: A Generative Part Seq2Seq Network for 3D Shapescited by 118
  18. 2020Cascaded Refinement Network for Point Cloud Completioncited by 112
  19. 2017ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scanscited by 103
  20. 2021Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networkscited by 101
  21. 2021Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networkscited by 98
  22. 2021Bidirectional Projection Network for Cross Dimension Scene Understandingcited by 95
  23. 2021Holistic 3D Scene Understanding from a Single Image with Implicit Representationcited by 94
  24. 2017High-Resolution Shape Completion Using Deep Neural Networks for Global Structure and Local Geometry Inferencecited by 87
  25. 2020MLCVNet: Multi-Level Context VoteNet for 3D Object Detectioncited by 80
  26. 20193D Instance Segmentation via Multi-Task Metric Learningcited by 79
  27. 2019SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scanscited by 77
  28. 2020DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolutioncited by 74
  29. 2020Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrievecited by 65
  30. 2019JSIS3D: Joint Semantic-Instance Segmentation of 3D Point Clouds with Multi-Task Pointwise Networks and Multi-Value Conditional Random Fieldscited by 62
  31. 2019StructureNet: Hierarchical Graph Networks for 3D Shape Generationcited by 59
  32. 2018Cooperative Holistic Scene Understanding: Unifying 3D Object, Layout, and Camera Pose Estimationcited by 51
  33. 2017O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysiscited by 47
  34. 2020SceneCAD: Predicting Object Alignments and Layouts in RGB-D Scanscited by 44
  35. 2021Joint Learning of 3D Shape Retrieval and Deformationcited by 41
  36. 2020RfD-Net: Point Scene Understanding by Semantic Instance Reconstructioncited by 36
  37. 2020SPSG: Self-Supervised Photometric Scene Generation from RGB-D Scanscited by 34
  38. 2020Vid2CAD: CAD Model Alignment using Multi-View Constraints from Videoscited by 33

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 7; 2024: 6; 2025: 3; 2026: 1.

20 papers.

Sort:

By area:

  1. 2022Compressible-composable NeRF via Rank-residual Decompositioncited by 92
  2. 2023Uncovering and Quantifying Social Biases in Code Generationcited by 25
  3. 2023Interactive Segment Anything NeRF with Feature Imitationcited by 24
  4. 2022Conditional DETR V2: Efficient Detection Transformer with Box Queriescited by 21
  5. 2022D-TensoRF: Tensorial Radiance Fields for Dynamic Scenescited by 19
  6. 2024ChatCam: Empowering Camera Control through Conversational AIcited by 15
  7. 2025Detection Based Part-level Articulated Object Reconstruction from Single RGBD Imagecited by 13
  8. 20243D Question Answering for City Scene Understandingcited by 10
  9. 2024Spectrum AUC Difference (SAUCD): Human-aligned 3D Shape Evaluationcited by 9
  10. 2024Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Samplingcited by 7
  11. 2023Uncovering and Categorizing Social Biases in Text-to-SQLcited by 5
  12. 2023LASA: Instance Reconstruction from Real Scans using A Large-scale Aligned Shape Annotation Datasetcited by 5
8 more
  1. 2023Parallel Sentence-Level Explanation Generation for Real-World Low-Resource Scenarioscited by 4
  2. 2023Real-time 3D Semantic Scene Completion Via Feature Aggregation and Conditioned Predictioncited by 3
  3. 2024Improving Long Text Understanding with Knowledge Distilled from Summarization Modelcited by 3
  4. 2023Shape Anchor Guided Holistic Indoor Scene Understandingcited by 2
  5. 2025Point-based Instance Completion with Scene Constraintscited by 1
  6. 20263D Point World Models: Point Completion Enables More Accurate Dynamics Learningcited by 1
  7. 2024DOCTR: Disentangled Object-Centric Transformer for Point Scene Understandingcited by 0
  8. 2025SRAM: Shape-Realism Alignment Metric for No Reference 3D Shape Evaluationcited by 0