# ICM-3D: Instantiated Category Modeling for 3D Instance Segmentation

arXiv: https://arxiv.org/abs/2108.11771 · HTML: https://mlsanity.com/paper/2108.11771/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 35 references that are in the map.
- First posted on arXiv 26 Aug 2021.

## Builds on (most cited first, 29 of 29)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation](https://mlsanity.com/paper/2004.01658/) (2020)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans](https://mlsanity.com/paper/1812.07003/) (2018)
- [Associative Embedding: End-to-End Learning for Joint Detection and Grouping](https://mlsanity.com/paper/1611.05424/) (2016)
- [OccuSeg: Occupancy-aware 3D Instance Segmentation](https://mlsanity.com/paper/2003.06537/) (2020)
- [SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation](https://mlsanity.com/paper/1711.08588/) (2017)
- [Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds](https://mlsanity.com/paper/1906.01140/) (2019)
- [GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud](https://mlsanity.com/paper/1812.03320/) (2018)
- [3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation](https://mlsanity.com/paper/2003.13867/) (2020)
- [Fully Convolutional Instance-aware Semantic Segmentation](https://mlsanity.com/paper/1611.07709/) (2016)
- [Associatively Segmenting Instances and Semantics in Point Clouds](https://mlsanity.com/paper/1902.09852/) (2019)
- [SSAP: Single-Shot Instance Segmentation With Affinity Pyramid](https://mlsanity.com/paper/1909.01616/) (2019)
- [Unseen Object Instance Segmentation for Robotic Environments](https://mlsanity.com/paper/2007.08073/) (2020)
- [3D Instance Segmentation via Multi-Task Metric Learning](https://mlsanity.com/paper/1906.08650/) (2019)
- [Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation](https://mlsanity.com/paper/2007.15157/) (2020)
- [Repulsion Loss: Detecting Pedestrians in a Crowd](https://mlsanity.com/paper/1711.07752/) (2017)
- [EasyLabel: A Semi-Automatic Pixel-wise Object Annotation Tool for Creating Robotic RGB-D Datasets](https://mlsanity.com/paper/1902.01626/) (2019)
- [Point Cloud Instance Segmentation using Probabilistic Embeddings](https://mlsanity.com/paper/1912.00145/) (2019)
- [RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks](https://mlsanity.com/paper/2106.15711/) (2021)
- [SceneCut: Joint Geometric and Object Segmentation for Indoor Scenes](https://mlsanity.com/paper/1709.07158/) (2017)

## Built on by (most cited first, 18 of 18)

- [VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking](https://mlsanity.com/paper/2303.11301/) (2023)
- [Stratified Transformer for 3D Point Cloud Segmentation](https://mlsanity.com/paper/2203.14508/) (2022)
- [Focal Sparse Convolutional Networks for 3D Object Detection](https://mlsanity.com/paper/2204.12463/) (2022)
- [Spherical Transformer for LiDAR-based 3D Recognition](https://mlsanity.com/paper/2303.12766/) (2023)
- [PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Models](https://mlsanity.com/paper/2212.01558/) (2022)
- [LargeKernel3D: Scaling up Kernels in 3D Sparse CNNs](https://mlsanity.com/paper/2206.10555/) (2022)
- [Masked Scene Contrast: A Scalable Framework for Unsupervised 3D Representation Learning](https://mlsanity.com/paper/2303.14191/) (2023)
- [Understanding Imbalanced Semantic Segmentation Through Neural Collapse](https://mlsanity.com/paper/2301.01100/) (2023)
- [Mask-Attention-Free Transformer for 3D Instance Segmentation](https://mlsanity.com/paper/2309.01692/) (2023)
- [PartSLIP++: Enhancing Low-Shot 3D Part Segmentation via Multi-View Instance Segmentation and Maximum Likelihood Estimation](https://mlsanity.com/paper/2312.03015/) (2023)
- [MaskGroup: Hierarchical Point Grouping and Masking for 3D Instance Segmentation](https://mlsanity.com/paper/2203.14662/) (2022)
- [UniGS: Unified Representation for Image Generation and Segmentation](https://mlsanity.com/paper/2312.01985/) (2023)
- [PartSTAD: 2D-to-3D Part Segmentation Task Adaptation](https://mlsanity.com/paper/2401.05906/) (2024)
- [Point2Pix: Photo-Realistic Point Cloud Rendering via Neural Radiance Fields](https://mlsanity.com/paper/2303.16482/) (2023)
- [A Review of Panoptic Segmentation for Mobile Mapping Point Clouds](https://mlsanity.com/paper/2304.13980/) (2023)
- [Rethinking Evaluation Metrics of Open-Vocabulary Segmentaion](https://mlsanity.com/paper/2311.03352/) (2023)
- [Window Normalization: Enhancing Point Cloud Understanding by Unifying Inconsistent Point Densities](https://mlsanity.com/paper/2212.02287/) (2022)
- [PinPoint3D: Fine-Grained 3D Part Segmentation from a Few Clicks](https://mlsanity.com/paper/2509.25970/) (2025)

## Abstract (arXiv)

Separating 3D point clouds into individual instances is an important task for 3D vision. It is challenging due to the unknown and varying number of instances in a scene. Existing deep learning based works focus on a two-step pipeline: first learn a feature embedding and then cluster the points. Such a two-step pipeline leads to disconnected intermediate objectives. In this paper, we propose an integrated reformulation of 3D instance segmentation as a per-point classification problem. We propose ICM-3D, a single-step method to segment 3D instances via instantiated categorization. The augmented category information is automatically constructed from 3D spatial positions. We conduct extensive experiments to verify the effectiveness of ICM-3D and show that it obtains inspiring performance across multiple frameworks, backbones and benchmarks.