# PointInst3D: Segmenting 3D Instances by Points arXiv: https://arxiv.org/abs/2204.11402 ยท HTML: https://mlsanity.com/paper/2204.11402/ ## Key facts - Cited by 16 papers in the map (data through 14 Oct 2026). - Builds on 38 of its 43 references that are in the map. - First posted on arXiv 25 Apr 2022. ## Builds on (most cited first, 30 of 38) - [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015) - [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017) - [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016) - [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017) - [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013) - [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019) - [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017) - [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019) - [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017) - [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019) - [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017) - [Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection](https://mlsanity.com/paper/1912.02424/) (2019) - [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018) - [Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs](https://mlsanity.com/paper/1711.09869/) (2017) - [PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation](https://mlsanity.com/paper/2004.01658/) (2020) - [Conditional Convolutions for Instance Segmentation](https://mlsanity.com/paper/2003.05664/) (2020) - [3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans](https://mlsanity.com/paper/1812.07003/) (2018) - [UnitBox: An Advanced Object Detection Network](https://mlsanity.com/paper/1608.01471/) (2016) - [OTA: Optimal Transport Assignment for Object Detection](https://mlsanity.com/paper/2103.14259/) (2021) - [OccuSeg: Occupancy-aware 3D Instance Segmentation](https://mlsanity.com/paper/2003.06537/) (2020) - [FCOS: A simple and strong anchor-free object detector](https://mlsanity.com/paper/2006.09214/) (2020) - [SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation](https://mlsanity.com/paper/1711.08588/) (2017) - [Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds](https://mlsanity.com/paper/1906.01140/) (2019) - [GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud](https://mlsanity.com/paper/1812.03320/) (2018) - [3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation](https://mlsanity.com/paper/2003.13867/) (2020) - [Hierarchical Aggregation for 3D Instance Segmentation](https://mlsanity.com/paper/2108.02350/) (2021) - [Semantic Instance Segmentation with a Discriminative Loss Function](https://mlsanity.com/paper/1708.02551/) (2017) - [Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks](https://mlsanity.com/paper/2108.07478/) (2021) - [PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things](https://mlsanity.com/paper/1903.01177/) (2019) - [Associatively Segmenting Instances and Semantics in Point Clouds](https://mlsanity.com/paper/1902.09852/) (2019) ## Built on by (most cited first, 16 of 16) - [Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance](https://mlsanity.com/paper/2312.10671/) (2023) - [ISBNet: a 3D Point Cloud Instance Segmentation Network with Instance-aware Sampling and Box-aware Dynamic Convolution](https://mlsanity.com/paper/2303.00246/) (2023) - [AGILE3D: Attention Guided Interactive Multi-object 3D Segmentation](https://mlsanity.com/paper/2306.00977/) (2023) - [Spherical Mask: Coarse-to-Fine 3D Point Cloud Instance Segmentation with Spherical Representation](https://mlsanity.com/paper/2312.11269/) (2023) - [3D-QueryIS: A Query-based Framework for 3D Instance Segmentation](https://mlsanity.com/paper/2211.09375/) (2022) - [Segment Any 3D Object with Language](https://mlsanity.com/paper/2404.02157/) (2024) - [MM-3DScene: 3D Scene Understanding by Customizing Masked Modeling with Informative-Preserved Reconstruction and Self-Distilled Consistency](https://mlsanity.com/paper/2212.09948/) (2022) - [GaPro: Box-Supervised 3D Point Cloud Instance Segmentation Using Gaussian Processes as Pseudo Labelers](https://mlsanity.com/paper/2307.13251/) (2023) - [MSTA3D: Multi-scale Twin-attention for 3D Instance Segmentation](https://mlsanity.com/paper/2411.01781/) (2024) - [Instance Consistency Regularization for Semi-Supervised 3D Instance Segmentation](https://mlsanity.com/paper/2406.16776/) (2024) - [ReScene4D: Temporally Consistent Semantic Instance Segmentation of Evolving Indoor 3D Scenes](https://mlsanity.com/paper/2601.11508/) (2026) - [EipFormer: Emphasizing Instance Positions in 3D Instance Segmentation](https://mlsanity.com/paper/2312.05602/) (2023) - [OSIS: Efficient One-stage Network for 3D Instance Segmentation](https://mlsanity.com/paper/2303.07011/) (2023) - [PSGformer: Enhancing 3D Point Cloud Instance Segmentation via Precise Semantic Guidance](https://mlsanity.com/paper/2307.07708/) (2023) - [DoReMi: Bridging 3D Domains via Topology-Aware Domain-Representation Mixture of Experts](https://mlsanity.com/paper/2511.11232/) (2025) - [STAR: A Spatial-Topology Aware Routing Framework for Generalizable 3D Scene Understanding](https://mlsanity.com/paper/2608.11699/) (2026) ## Abstract (arXiv) The current state-of-the-art methods in 3D instance segmentation typically involve a clustering step, despite the tendency towards heuristics, greedy algorithms, and a lack of robustness to the changes in data statistics. In contrast, we propose a fully-convolutional 3D point cloud instance segmentation method that works in a per-point prediction fashion. In doing so it avoids the challenges that clustering-based methods face: introducing dependencies among different tasks of the model. We find the key to its success is assigning a suitable target to each sampled point. Instead of the commonly used static or distance-based assignment strategies, we propose to use an Optimal Transport approach to optimally assign target masks to the sampled points according to the dynamic matching costs. Our approach achieves promising results on both ScanNet and S3DIS benchmarks. The proposed approach removes intertask dependencies and thus represents a simpler and more flexible 3D instance segmentation framework than other competing methods, while achieving improved segmentation accuracy.