# Point-based Instance Completion with Scene Constraints

arXiv: https://arxiv.org/abs/2504.05698 · HTML: https://mlsanity.com/paper/2504.05698/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 37 references that are in the map.
- First posted on arXiv 8 Apr 2025.

## Builds on (most cited first, 30 of 30)

- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [PCN: Point Completion Network](https://mlsanity.com/paper/1808.00671/) (2018)
- [Mask3D: Mask Transformer for 3D Semantic Instance Segmentation](https://mlsanity.com/paper/2210.03105/) (2022)
- [PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers](https://mlsanity.com/paper/2108.08839/) (2021)
- [SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer](https://mlsanity.com/paper/2108.04444/) (2021)
- [PF-Net: Point Fractal Network for 3D Point Cloud Completion](https://mlsanity.com/paper/2003.00410/) (2020)
- [Scan2CAD: Learning CAD Model Alignment in RGB-D Scans](https://mlsanity.com/paper/1811.11187/) (2018)
- [Morphing and Sampling Network for Dense Point Cloud Completion](https://mlsanity.com/paper/1912.00280/) (2019)
- [3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior](https://mlsanity.com/paper/2003.14052/) (2020)
- [Point Cloud Completion by Skip-attention Network with Hierarchical Folding](https://mlsanity.com/paper/2005.03871/) (2020)
- [Neural Kernel Surface Reconstruction](https://mlsanity.com/paper/2305.19590/) (2023)
- [PMP-Net: Point Cloud Completion by Learning Multi-step Point Moving Paths](https://mlsanity.com/paper/2012.03408/) (2020)
- [Dual Octree Graph Networks for Learning Adaptive Volumetric Shape Representations](https://mlsanity.com/paper/2205.02825/) (2022)
- [RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completion](https://mlsanity.com/paper/1903.00620/) (2019)
- [Semantic Scene Completion via Integrating Instances and Scene in-the-Loop](https://mlsanity.com/paper/2104.03640/) (2021)
- [PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awareness](https://mlsanity.com/paper/2312.02158/) (2023)
- [End-to-End CAD Model Retrieval and 9DoF Alignment in 3D Scans](https://mlsanity.com/paper/1906.04201/) (2019)
- [RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction](https://mlsanity.com/paper/2011.14744/) (2020)
- [RevealNet: Seeing Behind Objects in RGB-D Scans](https://mlsanity.com/paper/1904.12012/) (2019)
- [Point Scene Understanding via Disentangled Instance Mesh Reconstruction](https://mlsanity.com/paper/2203.16832/) (2022)
- [CAD-Deform: Deformable Fitting of CAD Models to 3D Scans](https://mlsanity.com/paper/2007.11965/) (2020)
- [SCARP: 3D Shape Completion in ARbitrary Poses for Improved Grasping](https://mlsanity.com/paper/2301.07213/) (2023)
- [CVSformer: Cross-View Synthesis Transformer for Semantic Scene Completion](https://mlsanity.com/paper/2307.07938/) (2023)
- [Unleashing Network Potentials for Semantic Scene Completion](https://mlsanity.com/paper/2403.07560/) (2024)
- [Diverse Shape Completion via Style Modulated Generative Adversarial Networks](https://mlsanity.com/paper/2311.11184/) (2023)

## Built on by (most cited first, 1 of 1)

- [3D Point World Models: Point Completion Enables More Accurate Dynamics Learning](https://mlsanity.com/paper/2607.00148/) (2026)

## Abstract (arXiv)

Recent point-based object completion methods have demonstrated the ability to accurately recover the missing geometry of partially observed objects. However, these approaches are not well-suited for completing objects within a scene, as they do not consider known scene constraints (e.g., other observed surfaces) in their completions and further expect the partial input to be in a canonical coordinate system, which does not hold for objects within scenes. While instance scene completion methods have been proposed for completing objects within a scene, they lag behind point-based object completion methods in terms of object completion quality and still do not consider known scene constraints during completion. To overcome these limitations, we propose a point cloud-based instance completion model that can robustly complete objects at arbitrary scales and pose in the scene. To enable reasoning at the scene level, we introduce a sparse set of scene constraints represented as point clouds and integrate them into our completion model via a cross-attention mechanism. To evaluate the instance scene completion task on indoor scenes, we further build a new dataset called ScanWCF, which contains labeled partial scans as well as aligned ground truth scene completions that are watertight and collision-free. Through several experiments, we demonstrate that our method achieves improved fidelity to partial scans, higher completion quality, and greater plausibility over existing state-of-the-art methods.