# ISBNet: a 3D Point Cloud Instance Segmentation Network with Instance-aware Sampling and Box-aware Dynamic Convolution

arXiv: https://arxiv.org/abs/2303.00246 · HTML: https://mlsanity.com/paper/2303.00246/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 51 references that are in the map.
- First posted on arXiv 1 Mar 2023.

## Builds on (most cited first, 30 of 37)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs](https://mlsanity.com/paper/1711.09869/) (2017)
- [Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation](https://mlsanity.com/paper/2206.02777/) (2022)
- [PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation](https://mlsanity.com/paper/2004.01658/) (2020)
- [Conditional Convolutions for Instance Segmentation](https://mlsanity.com/paper/2003.05664/) (2020)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Mask Scoring R-CNN](https://mlsanity.com/paper/1903.00241/) (2019)
- [3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans](https://mlsanity.com/paper/1812.07003/) (2018)
- [SoftGroup for 3D Instance Segmentation on Point Clouds](https://mlsanity.com/paper/2203.01509/) (2022)
- [OccuSeg: Occupancy-aware 3D Instance Segmentation](https://mlsanity.com/paper/2003.06537/) (2020)
- [SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation](https://mlsanity.com/paper/1711.08588/) (2017)
- [Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds](https://mlsanity.com/paper/1906.01140/) (2019)
- [DETRs with Hybrid Matching](https://mlsanity.com/paper/2207.13080/) (2022)
- [GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud](https://mlsanity.com/paper/1812.03320/) (2018)
- [3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation](https://mlsanity.com/paper/2003.13867/) (2020)
- [Hierarchical Aggregation for 3D Instance Segmentation](https://mlsanity.com/paper/2108.02350/) (2021)
- [Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks](https://mlsanity.com/paper/2108.07478/) (2021)
- [DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution](https://mlsanity.com/paper/2011.13328/) (2020)
- [JSIS3D: Joint Semantic-Instance Segmentation of 3D Point Clouds with Multi-Task Pointwise Networks and Multi-Value Conditional Random Fields](https://mlsanity.com/paper/1904.00699/) (2019)

## Built on by (most cited first, 30 of 53)

- [Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance](https://mlsanity.com/paper/2312.10671/) (2023)
- [OneFormer3D: One Transformer for Unified Point Cloud Segmentation](https://mlsanity.com/paper/2311.14405/) (2023)
- [Spherical Mask: Coarse-to-Fine 3D Point Cloud Instance Segmentation with Spherical Representation](https://mlsanity.com/paper/2312.11269/) (2023)
- [Mosaic3D: Foundation Dataset and Model for Open-Vocabulary 3D Segmentation](https://mlsanity.com/paper/2502.02548/) (2025)
- [SANeRF-HQ: Segment Anything for NeRF in High Quality](https://mlsanity.com/paper/2312.01531/) (2023)
- [A Unified Framework for 3D Scene Understanding](https://mlsanity.com/paper/2407.03263/) (2024)
- [Any3DIS: Class-Agnostic 3D Instance Segmentation by 2D Mask Tracking](https://mlsanity.com/paper/2411.16183/) (2024)
- [TreeLearn: A deep learning method for segmenting individual trees from ground-based LiDAR forest point clouds](https://mlsanity.com/paper/2309.08471/) (2023)
- [Scalable 3D Panoptic Segmentation As Superpoint Graph Clustering](https://mlsanity.com/paper/2401.06704/) (2024)
- [Segment Any 3D Object with Language](https://mlsanity.com/paper/2404.02157/) (2024)
- [Trace3D: Consistent Segmentation Lifting via Gaussian Instance Tracing](https://mlsanity.com/paper/2508.03227/) (2025)
- [SGIFormer: Semantic-guided and Geometric-enhanced Interleaving Transformer for 3D Instance Segmentation](https://mlsanity.com/paper/2407.11564/) (2024)
- [OnlineAnySeg: Online Zero-Shot 3D Segmentation by Visual Foundation Model Guided 2D Mask Merging](https://mlsanity.com/paper/2503.01309/) (2025)
- [Symbol as Points: Panoptic Symbol Spotting via Point-based Representation](https://mlsanity.com/paper/2401.10556/) (2024)
- [Relation3D: Enhancing Relation Modeling for Point Cloud Instance Segmentation](https://mlsanity.com/paper/2506.17891/) (2025)
- [Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception](https://mlsanity.com/paper/2508.11256/) (2025)
- [When 3D Bounding-Box Meets SAM: Point Cloud Instance Segmentation with Weak-and-Noisy Supervision](https://mlsanity.com/paper/2309.00828/) (2023)
- [BSNet: Box-Supervised Simulation-assisted Mean Teacher for 3D Instance Segmentation](https://mlsanity.com/paper/2403.15019/) (2024)
- [Details Matter for Indoor Open-vocabulary 3D Instance Segmentation](https://mlsanity.com/paper/2507.23134/) (2025)
- [GaPro: Box-Supervised 3D Point Cloud Instance Segmentation Using Gaussian Processes as Pseudo Labelers](https://mlsanity.com/paper/2307.13251/) (2023)
- [OpenScan: A Benchmark for Generalized Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2408.11030/) (2024)
- [MSTA3D: Multi-scale Twin-attention for 3D Instance Segmentation](https://mlsanity.com/paper/2411.01781/) (2024)
- [CompetitorFormer: Competitor Transformer for 3D Instance Segmentation](https://mlsanity.com/paper/2411.14179/) (2024)
- [Sketchy Bounding-box Supervision for 3D Instance Segmentation](https://mlsanity.com/paper/2505.16399/) (2025)
- [All in One: Visual-Description-Guided Unified Point Cloud Segmentation](https://mlsanity.com/paper/2507.05211/) (2025)
- [Zero-Shot Dual-Path Integration Framework for Open-Vocabulary 3D Instance Segmentation](https://mlsanity.com/paper/2408.08591/) (2024)
- [OE3DIS: Open-Ended 3D Point Cloud Instance Segmentation](https://mlsanity.com/paper/2408.11747/) (2024)
- [Beyond the Final Layer: Hierarchical Query Fusion Transformer with Agent-Interpolation Initialization for 3D Instance Segmentation](https://mlsanity.com/paper/2502.04139/) (2025)
- [OpenTrack3D: Towards Accurate and Generalizable Open-Vocabulary 3D Instance Segmentation](https://mlsanity.com/paper/2512.03532/) (2025)
- [LaSSM: Efficient Semantic-Spatial Query Decoding via Local Aggregation and State Space Models for 3D Instance Segmentation](https://mlsanity.com/paper/2602.11007/) (2026)

## Abstract (arXiv)

Existing 3D instance segmentation methods are predominated by the bottom-up design -- manually fine-tuned algorithm to group points into clusters followed by a refinement network. However, by relying on the quality of the clusters, these methods generate susceptible results when (1) nearby objects with the same semantic class are packed together, or (2) large objects with loosely connected regions. To address these limitations, we introduce ISBNet, a novel cluster-free method that represents instances as kernels and decodes instance masks via dynamic convolution. To efficiently generate high-recall and discriminative kernels, we propose a simple strategy named Instance-aware Farthest Point Sampling to sample candidates and leverage the local aggregation layer inspired by PointNet++ to encode candidate features. Moreover, we show that predicting and leveraging the 3D axis-aligned bounding boxes in the dynamic convolution further boosts performance. Our method set new state-of-the-art results on ScanNetV2 (55.9), S3DIS (60.8), and STPLS3D (49.2) in terms of AP and retains fast inference time (237ms per scene on ScanNetV2). The source code and trained models are available at https://github.com/VinAIResearch/ISBNet.