# MaskGroup: Hierarchical Point Grouping and Masking for 3D Instance Segmentation

arXiv: https://arxiv.org/abs/2203.14662 · HTML: https://mlsanity.com/paper/2203.14662/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 31 references that are in the map.
- First posted on arXiv 28 Mar 2022.

## Builds on (most cited first, 25 of 25)

- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation](https://mlsanity.com/paper/2004.01658/) (2020)
- [3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera](https://mlsanity.com/paper/1910.02527/) (2019)
- [3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans](https://mlsanity.com/paper/1812.07003/) (2018)
- [OccuSeg: Occupancy-aware 3D Instance Segmentation](https://mlsanity.com/paper/2003.06537/) (2020)
- [Acquisition of Localization Confidence for Accurate Object Detection](https://mlsanity.com/paper/1807.11590/) (2018)
- [SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation](https://mlsanity.com/paper/1711.08588/) (2017)
- [Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds](https://mlsanity.com/paper/1906.01140/) (2019)
- [3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation](https://mlsanity.com/paper/2003.13867/) (2020)
- [Hierarchical Aggregation for 3D Instance Segmentation](https://mlsanity.com/paper/2108.02350/) (2021)
- [GS3D: An Efficient 3D Object Detection Framework for Autonomous Driving](https://mlsanity.com/paper/1903.10955/) (2019)
- [Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks](https://mlsanity.com/paper/2108.07478/) (2021)
- [PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things](https://mlsanity.com/paper/1903.01177/) (2019)
- [Associatively Segmenting Instances and Semantics in Point Clouds](https://mlsanity.com/paper/1902.09852/) (2019)
- [3D Instance Segmentation via Multi-Task Metric Learning](https://mlsanity.com/paper/1906.08650/) (2019)
- [DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution](https://mlsanity.com/paper/2011.13328/) (2020)
- [Point Cloud Instance Segmentation using Probabilistic Embeddings](https://mlsanity.com/paper/1912.00145/) (2019)
- [Learning Gaussian Instance Segmentation in Point Clouds](https://mlsanity.com/paper/2007.09860/) (2020)
- [Learning and Memorizing Representative Prototypes for 3D Point Cloud Semantic and Instance Segmentation](https://mlsanity.com/paper/2001.01349/) (2020)
- [ICM-3D: Instantiated Category Modeling for 3D Instance Segmentation](https://mlsanity.com/paper/2108.11771/) (2021)
- [Hierarchy Denoising Recursive Autoencoders for 3D Scene Layout Prediction](https://mlsanity.com/paper/1903.03757/) (2019)
- [An Empirical Study of Adder Neural Networks for Object Detection](https://mlsanity.com/paper/2112.13608/) (2021)

## Built on by (most cited first, 30 of 30)

- [Mask3D: Mask Transformer for 3D Semantic Instance Segmentation](https://mlsanity.com/paper/2210.03105/) (2022)
- [Compressible-composable NeRF via Rank-residual Decomposition](https://mlsanity.com/paper/2205.14870/) (2022)
- [Mask-Attention-Free Transformer for 3D Instance Segmentation](https://mlsanity.com/paper/2309.01692/) (2023)
- [Top-Down Beats Bottom-Up in 3D Instance Segmentation](https://mlsanity.com/paper/2302.02871/) (2023)
- [Divide and Conquer: 3D Point Cloud Instance Segmentation With Point-Wise Binarization](https://mlsanity.com/paper/2207.11209/) (2022)
- [Uncovering and Quantifying Social Biases in Code Generation](https://mlsanity.com/paper/2305.15377/) (2023)
- [Interactive Segment Anything NeRF with Feature Imitation](https://mlsanity.com/paper/2305.16233/) (2023)
- [Spherical Mask: Coarse-to-Fine 3D Point Cloud Instance Segmentation with Spherical Representation](https://mlsanity.com/paper/2312.11269/) (2023)
- [Automated forest inventory: analysis of high-density airborne LiDAR point clouds with 3D deep learning](https://mlsanity.com/paper/2312.15084/) (2023)
- [Point Scene Understanding via Disentangled Instance Mesh Reconstruction](https://mlsanity.com/paper/2203.16832/) (2022)
- [3D-QueryIS: A Query-based Framework for 3D Instance Segmentation](https://mlsanity.com/paper/2211.09375/) (2022)
- [ForestFormer3D: A Unified Framework for End-to-End Segmentation of Forest LiDAR 3D Point Clouds](https://mlsanity.com/paper/2506.16991/) (2025)
- [Relation3D: Enhancing Relation Modeling for Point Cloud Instance Segmentation](https://mlsanity.com/paper/2506.17891/) (2025)
- [Uni3DL: Unified Model for 3D and Language Understanding](https://mlsanity.com/paper/2312.03026/) (2023)
- [BSNet: Box-Supervised Simulation-assisted Mean Teacher for 3D Instance Segmentation](https://mlsanity.com/paper/2403.15019/) (2024)
- [Towards accurate instance segmentation in large-scale LiDAR point clouds](https://mlsanity.com/paper/2307.02877/) (2023)
- [PointWavelet: Learning in Spectral Domain for 3D Point Cloud Analysis](https://mlsanity.com/paper/2302.05201/) (2023)
- [Vocabulary-Free 3D Instance Segmentation with Vision and Language Assistant](https://mlsanity.com/paper/2408.10652/) (2024)
- [MSTA3D: Multi-scale Twin-attention for 3D Instance Segmentation](https://mlsanity.com/paper/2411.01781/) (2024)
- [CompetitorFormer: Competitor Transformer for 3D Instance Segmentation](https://mlsanity.com/paper/2411.14179/) (2024)
- [Hi-LSplat: Hierarchical 3D Language Gaussian Splatting](https://mlsanity.com/paper/2506.06822/) (2025)
- [Real-time 3D Semantic Scene Completion Via Feature Aggregation and Conditioned Prediction](https://mlsanity.com/paper/2303.10967/) (2023)
- [Weakly Supervised 3D Instance Segmentation without Instance-level Annotations](https://mlsanity.com/paper/2308.01721/) (2023)
- [Beyond the Final Layer: Hierarchical Query Fusion Transformer with Agent-Interpolation Initialization for 3D Instance Segmentation](https://mlsanity.com/paper/2502.04139/) (2025)
- [Open-Vocabulary Functional 3D Human-Scene Interaction Generation](https://mlsanity.com/paper/2601.20835/) (2026)
- [SegmentAnyTreeV2: Scaling Transformer-Based Tree Instance Segmentation Across Sensors, Platforms, and Forests](https://mlsanity.com/paper/2606.08206/) (2026)
- [BEEP3D: Box-Supervised End-to-End Pseudo-Mask Generation for 3D Instance Segmentation](https://mlsanity.com/paper/2510.12182/) (2025)
- [Hierarchical Image-Guided 3D Point Cloud Segmentation in Industrial Scenes via Multi-View Bayesian Fusion](https://mlsanity.com/paper/2512.06882/) (2025)
- [3D-CDRGP: Towards Cross-Device Robotic Grasping Policy in 3D Open World](https://mlsanity.com/paper/2411.18133/) (2024)
- [Geometry-Guided Semantic Reconstruction for 3D Instance Segmentation](https://mlsanity.com/paper/2610.16122/) (2026)

## Abstract (arXiv)

This paper studies the 3D instance segmentation problem, which has a variety of real-world applications such as robotics and augmented reality. Since the surroundings of 3D objects are of high complexity, the separating of different objects is very difficult. To address this challenging problem, we propose a novel framework to group and refine the 3D instances. In practice, we first learn an offset vector for each point and shift it to its predicted instance center. To better group these points, we propose a Hierarchical Point Grouping algorithm to merge the centrally aggregated points progressively. All points are grouped into small clusters, which further gradually undergo another clustering procedure to merge into larger groups. These multi-scale groups are exploited for instance prediction, which is beneficial for predicting instances with different scales. In addition, a novel MaskScoreNet is developed to produce binary point masks of these groups for further refining the segmentation results. Extensive experiments conducted on the ScanNetV2 and S3DIS benchmarks demonstrate the effectiveness of the proposed method. For instance, our approach achieves a 66.4% mAP with the 0.5 IoU threshold on the ScanNetV2 test set, which is 1.9% higher than the state-of-the-art method.