# SAM-guided Graph Cut for 3D Instance Segmentation

arXiv: https://arxiv.org/abs/2312.08372 · HTML: https://mlsanity.com/paper/2312.08372/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 67 references that are in the map.
- First posted on arXiv 13 Dec 2023.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes](https://mlsanity.com/paper/2308.11417/) (2023)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [OpenScene: 3D Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2211.15654/) (2022)
- [In-Place Scene Labelling and Understanding with Implicit Scene Representation](https://mlsanity.com/paper/2103.15875/) (2021)
- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation](https://mlsanity.com/paper/2004.01658/) (2020)
- [SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters](https://mlsanity.com/paper/1803.11527/) (2018)
- [Language-Grounded Indoor 3D Semantic Segmentation in the Wild](https://mlsanity.com/paper/2204.07761/) (2022)
- [Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering](https://mlsanity.com/paper/2109.01847/) (2021)
- [3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans](https://mlsanity.com/paper/1812.07003/) (2018)
- [SoftGroup for 3D Instance Segmentation on Point Clouds](https://mlsanity.com/paper/2203.01509/) (2022)
- [SEGCloud: Semantic Segmentation of 3D Point Clouds](https://mlsanity.com/paper/1710.07563/) (2017)

## Built on by (most cited first, 30 of 35)

- [Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance](https://mlsanity.com/paper/2312.10671/) (2023)
- [PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyond](https://mlsanity.com/paper/2504.11451/) (2025)
- [Beyond Bare Queries: Open-Vocabulary Object Grounding with 3D Scene Graph](https://mlsanity.com/paper/2406.07113/) (2024)
- [DarkSAM: Fooling Segment Anything Model to Segment Nothing](https://mlsanity.com/paper/2409.17874/) (2024)
- [GaussianCut: Interactive segmentation via graph cut for 3D Gaussian Splatting](https://mlsanity.com/paper/2411.07555/) (2024)
- [ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splatting](https://mlsanity.com/paper/2507.15454/) (2025)
- [Any3DIS: Class-Agnostic 3D Instance Segmentation by 2D Mask Tracking](https://mlsanity.com/paper/2411.16183/) (2024)
- [Trace3D: Consistent Segmentation Lifting via Gaussian Instance Tracing](https://mlsanity.com/paper/2508.03227/) (2025)
- [PointSeg: A Training-Free Paradigm for 3D Scene Segmentation via Foundation Models](https://mlsanity.com/paper/2403.06403/) (2024)
- [PartUV: Part-Based UV Unwrapping of 3D Meshes](https://mlsanity.com/paper/2511.16659/) (2025)
- [LogoSP: Local-global Grouping of Superpoints for Unsupervised Semantic Segmentation of 3D Point Clouds](https://mlsanity.com/paper/2506.07857/) (2025)
- [Details Matter for Indoor Open-vocabulary 3D Instance Segmentation](https://mlsanity.com/paper/2507.23134/) (2025)
- [LiDAR Remote Sensing Meets Weak Supervision: Concepts, Methods, and Perspectives](https://mlsanity.com/paper/2503.18384/) (2025)
- [AutoInst: Automatic Instance-Based Segmentation of LiDAR 3D Scans](https://mlsanity.com/paper/2403.16318/) (2024)
- [OE3DIS: Open-Ended 3D Point Cloud Instance Segmentation](https://mlsanity.com/paper/2408.11747/) (2024)
- [SA3DIP: Segment Any 3D Instance with Potential 3D Priors](https://mlsanity.com/paper/2411.03819/) (2024)
- [E-SAM: Training-Free Segment Every Entity Model](https://mlsanity.com/paper/2503.12094/) (2025)
- [OpenTrack3D: Towards Accurate and Generalizable Open-Vocabulary 3D Instance Segmentation](https://mlsanity.com/paper/2512.03532/) (2025)
- [SGS-3D: High-Fidelity 3D Instance Segmentation via Reliable Semantic Mask Splitting and Growing](https://mlsanity.com/paper/2509.05144/) (2025)
- [Towards 3D Objectness Learning in an Open World](https://mlsanity.com/paper/2510.17686/) (2025)
- [EZ-SP: Fast and Lightweight Superpoint-Based 3D Segmentation](https://mlsanity.com/paper/2512.00385/) (2025)
- [You Only Click Once: Single Point Weakly Supervised 3D Instance Segmentation for Autonomous Driving](https://mlsanity.com/paper/2502.19698/) (2025)
- [MV3DIS: Multi-View Mask Matching via 3D Guides for Zero-Shot 3D Instance Segmentation](https://mlsanity.com/paper/2604.08916/) (2026)
- [EvObj: Learning Evolving Object-centric Representations for 3D Instance Segmentation without Scene Supervision](https://mlsanity.com/paper/2605.13152/) (2026)
- [TMI: Text-to-Image Meets Image-to-Image for Complementary Data Synthesis to Boost Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2607.08201/) (2026)
- [SAM-V: Geometry-Aware Segment Anything for Multi-View Instance Segmentation](https://mlsanity.com/paper/2609.25490/) (2026)
- [Automating Infrastructure Surveying: A Framework for Geometric Measurements and Compliance Assessment Using Point Cloud Data](https://mlsanity.com/paper/2505.05752/) (2025)
- [Unleashing the Multi-View Fusion Potential: Noise Correction in VLM for Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2506.22817/) (2025)
- [Class-agnostic 3D Segmentation by Granularity-Consistent Automatic 2D Mask Tracking](https://mlsanity.com/paper/2511.00785/) (2025)
- [ASSIST-3D: Adapted Scene Synthesis for Class-Agnostic 3D Instance Segmentation](https://mlsanity.com/paper/2512.09364/) (2025)

## Abstract (arXiv)

This paper addresses the challenge of 3D instance segmentation by simultaneously leveraging 3D geometric and multi-view image information. Many previous works have applied deep learning techniques to 3D point clouds for instance segmentation. However, these methods often failed to generalize to various types of scenes due to the scarcity and low-diversity of labeled 3D point cloud data. Some recent works have attempted to lift 2D instance segmentations to 3D within a bottom-up framework. The inconsistency in 2D instance segmentations among views can substantially degrade the performance of 3D segmentation. In this work, we introduce a novel 3D-to-2D query framework to effectively exploit 2D segmentation models for 3D instance segmentation. Specifically, we pre-segment the scene into several superpoints in 3D, formulating the task into a graph cut problem. The superpoint graph is constructed based on 2D segmentation models, where node features are obtained from multi-view image features and edge weights are computed based on multi-view segmentation results, enabling the better generalization ability. To process the graph, we train a graph neural network using pseudo 3D labels from 2D segmentation models. Experimental results on the ScanNet, ScanNet++ and KITTI-360 datasets demonstrate that our method achieves robust segmentation performance and can generalize across different types of scenes. Our project page is available at https://zju3dv.github.io/sam_graph.