Associatively Segmenting Instances and Semantics in Point Clouds
Not on the map. Why?
Abstract (arXiv)
A 3D point cloud describes the real scene precisely and intuitively.To date how to segment diversified elements in such an informative 3D scene is rarely discussed. In this paper, we first introduce a simple and flexible framework to segment instances and semantics in point clouds simultaneously. Then, we propose two approaches which make the two tasks take advantage of each other, leading to a win-win situation. Specifically, we make instance segmentation benefit from semantic segmentation through learning semantic-aware point-level instance embedding. Meanwhile, semantic features of the points belonging to the same instance are fused together to make more accurate per-point semantic predictions. Our method largely outperforms the state-of-the-art method in 3D instance segmentation along with a significant improvement in 3D semantic segmentation. Code has been made available at: https://github.com/WXinlong/ASIS.
Key facts. Cited by 86 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Feb 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 21; 2022: 27; 2023: 19; 2024: 8; 2025: 7; 2026: 4.
86 papers.
Sort:
By area:
- 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
- 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 299
- 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
- 2022SoftGroup for 3D Instance Segmentation on Point Cloudscited by 197
- 2021Hierarchical Aggregation for 3D Instance Segmentationcited by 126
- 2022PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Modelscited by 117
- 2021Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentationcited by 107
- 2021Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networkscited by 98
- 2021Exploiting Edge-Oriented Reasoning for 3D Point-based Scene Graph Analysiscited by 47
- 2021HVPR: Hybrid Voxel-Point Representation for Single-stage 3D Object Detectioncited by 47
- 2022Dynamic 3D Scene Analysis by Point Cloud Accumulationcited by 47
- 2023Mask-Attention-Free Transformer for 3D Instance Segmentationcited by 45
38 more
- 2023PartSLIP++: Enhancing Low-Shot 3D Part Segmentation via Multi-View Instance Segmentation and Maximum Likelihood Estimationcited by 37
- 2023AGILE3D: Attention Guided Interactive Multi-object 3D Segmentationcited by 33
- 2023Top-Down Beats Bottom-Up in 3D Instance Segmentationcited by 31
- 2021Deep Learning Based 3D Segmentation: A Surveycited by 30
- 2022MaskGroup: Hierarchical Point Grouping and Masking for 3D Instance Segmentationcited by 29
- 2022Divide and Conquer: 3D Point Cloud Instance Segmentation With Point-Wise Binarizationcited by 26
- 2023UrbanBIS: a Large-scale Benchmark for Fine-grained Urban Building Instance Segmentationcited by 24
- 2021Boundary-Aware Geometric Encoding for Semantic Segmentation of Point Cloudscited by 23
- 2022Interactive Object Segmentation in 3D Point Cloudscited by 23
- 2023Automated forest inventory: analysis of high-density airborne LiDAR point clouds with 3D deep learningcited by 23
- 2021Omni-supervised Point Cloud Segmentation via Gradual Receptive Field Component Reasoningcited by 22
- 2021DRINet: A Dual-Representation Iterative Learning Network for Point Cloud Segmentationcited by 22
- 2022Sparse Cross-scale Attention Network for Efficient LiDAR Panoptic Segmentationcited by 22
- 2021ICM-3D: Instantiated Category Modeling for 3D Instance Segmentationcited by 18
- 2024PartSTAD: 2D-to-3D Part Segmentation Task Adaptationcited by 18
- 2023Panoptic Vision-Language Feature Fieldscited by 17
- 2022PointInst3D: Segmenting 3D Instances by Pointscited by 16
- 2021Point Cloud Instance Segmentation with Semi-supervised Bounding-Box Miningcited by 15
- 2022Gaussian Radar Transformer for Semantic Segmentation in Noisy Radar Datacited by 15
- 2022Scalable SoftGroup for 3D Instance Segmentation on Point Cloudscited by 14
- 20223D-QueryIS: A Query-based Framework for 3D Instance Segmentationcited by 14
- 2023Joint Representation Learning for Text and 3D Point Cloudcited by 14
- 2024Spot-Compose: A Framework for Open-Vocabulary Object Retrieval and Drawer Manipulation in Point Cloudscited by 14
- 2022Learning Inter-Superpoint Affinity for Weakly Supervised 3D Instance Segmentationcited by 13
- 20213D Instance Segmentation of MVS Buildingscited by 12
- 2022NeuralBF: Neural Bilateral Filtering for Top-down Instance Segmentation on Point Cloudscited by 12
- 2025ForestFormer3D: A Unified Framework for End-to-End Segmentation of Forest LiDAR 3D Point Cloudscited by 12
- 2021LatticeNet: Fast Spatio-Temporal Point Cloud Segmentation Using Permutohedral Latticescited by 11
- 2022LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Networkcited by 11
- 2023Volumetric Semantically Consistent 3D Panoptic Mappingcited by 10
- 2021HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensorcited by 9
- 2023Radar Instance Transformer: Reliable Moving Instance Segmentation in Sparse Radar Point Cloudscited by 9
- 2023A Review of Panoptic Segmentation for Mobile Mapping Point Cloudscited by 8
- 2023Towards accurate instance segmentation in large-scale LiDAR point cloudscited by 8
- 2022PST: Plant segmentation transformer for 3D point clouds of rapeseed plants at the podding stagecited by 6
- 2023FreePoint: Unsupervised Point Cloud Instance Segmentationcited by 6
- 2021Hierarchical Lov'asz Embeddings for Proposal-free Panoptic Segmentationcited by 4
- 2022Primitive3D: 3D Object Dataset Synthesis from Randomly Assembled Primitivescited by 4
50 most cited of 86 All 86 (JSON)
- 2026PlantC2USeg: Cross-Scale Consistent Pre-Training for Few-Shot Unified Plant Point Cloud Segmentation2 Sep 2026
- 2026SegmentAnyTreeV2: Scaling Transformer-Based Tree Instance Segmentation Across Sensors, Platforms, and Forests6 Jun 2026
- 2026Joint 2D-3D Segmentation and Association in Street-level Imaging26 May 2026
- 2026SegVGGT: Joint 3D Reconstruction and Instance Segmentation from Multi-View Images20 Mar 2026
- 2025PinPoint3D: Fine-Grained 3D Part Segmentation from a Few Clicks30 Sep 2025
- 2025OVSeg3R: Learn Open-vocabulary Instance Segmentation from 2D via 3D Reconstruction28 Sep 2025
- 2025SegDINO3D: 3D Instance Segmentation Empowered by Both Image-Level and Object-Level 2D Features19 Sep 2025
- 2025ForestFormer3D: A Unified Framework for End-to-End Segmentation of Forest LiDAR 3D Point Clouds20 Jun 2025
- 2025SPPSFormer: High-quality Superpoint-based Transformer for Roof Plane Instance Segmentation from Point Clouds30 May 2025
- 2025RAZER: Robust Accelerated Zero-Shot 3D Open-Vocabulary Panoptic Reconstruction with Spatio-Temporal Aggregation21 May 2025
- 2025Beyond the Final Layer: Hierarchical Query Fusion Transformer with Agent-Interpolation Initialization for 3D Instance Segmentation6 Feb 2025
- 2024ProtoSeg: A Prototype-Based Point Cloud Instance Segmentation Method3 Oct 2024
38 more
- 2024Rethinking Attention Module Design for Point Cloud Analysis27 Jul 2024
- 2024Improved Block Merging for 3D Point Cloud Instance Segmentation9 Jul 2024
- 2024Joint prototype and coefficient prediction for 3D instance segmentation9 Jul 2024
- 2024Deep Learning-Based 3D Instance and Semantic Segmentation: A Review19 Jun 2024
- 2024Spot-Compose: A Framework for Open-Vocabulary Object Retrieval and Drawer Manipulation in Point Clouds18 Apr 2024
- 2024Advancements in Point Cloud-Based 3D Defect Detection and Classification for Industrial Systems: A Comprehensive Survey20 Feb 2024
- 2024PartSTAD: 2D-to-3D Part Segmentation Task Adaptation11 Jan 2024
- 2023Automated forest inventory: analysis of high-density airborne LiDAR point clouds with 3D deep learning22 Dec 2023
- 2023EipFormer: Emphasizing Instance Positions in 3D Instance Segmentation9 Dec 2023
- 2023PartSLIP++: Enhancing Low-Shot 3D Part Segmentation via Multi-View Instance Segmentation and Maximum Likelihood Estimation5 Dec 2023
- 2023Radar Instance Transformer: Reliable Moving Instance Segmentation in Sparse Radar Point Clouds28 Sep 2023
- 2023Volumetric Semantically Consistent 3D Panoptic Mapping26 Sep 2023
- 2023Panoptic Vision-Language Feature Fields11 Sep 2023
- 2023A boundary-aware point clustering approach in Euclidean and embedding spaces for roof plane segmentation7 Sep 2023
- 2023Mask-Attention-Free Transformer for 3D Instance Segmentation4 Sep 2023
- 2023Weakly Supervised 3D Instance Segmentation without Instance-level Annotations3 Aug 2023
- 2023PSGformer: Enhancing 3D Point Cloud Instance Segmentation via Precise Semantic Guidance15 Jul 2023
- 2023Towards accurate instance segmentation in large-scale LiDAR point clouds6 Jul 2023
- 2023AGILE3D: Attention Guided Interactive Multi-object 3D Segmentation1 Jun 2023
- 2023FreePoint: Unsupervised Point Cloud Instance Segmentation11 May 2023
- 2023UrbanBIS: a Large-scale Benchmark for Fine-grained Urban Building Instance Segmentation4 May 2023
- 2023A Review of Panoptic Segmentation for Mobile Mapping Point Clouds27 Apr 2023
- 2023Small but Mighty: Enhancing 3D Point Clouds Semantic Segmentation with U-Next Framework3 Apr 2023
- 2023OSIS: Efficient One-stage Network for 3D Instance Segmentation13 Mar 2023
- 2023Top-Down Beats Bottom-Up in 3D Instance Segmentation6 Feb 2023
- 2023Joint Representation Learning for Text and 3D Point Cloud18 Jan 2023
- 2022Gaussian Radar Transformer for Semantic Segmentation in Noisy Radar Data7 Dec 2022
- 2022PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Models3 Dec 2022
- 20223D-QueryIS: A Query-based Framework for 3D Instance Segmentation17 Nov 2022
- 2022Joint Data and Feature Augmentation for Self-Supervised Representation Learning on Point Clouds2 Nov 2022
- 2022Seg&Struct: The Interplay Between Part Segmentation and Structure Inference for 3D Shape Parsing1 Nov 2022
- 2022Instance Segmentation with Cross-Modal Consistency14 Oct 2022
- 2022Learning Inter-Superpoint Affinity for Weakly Supervised 3D Instance Segmentation11 Oct 2022
- 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentation6 Oct 2022
- 2022Scalable SoftGroup for 3D Instance Segmentation on Point Clouds17 Sep 2022
- 2022Collaborative Propagation on Multiple Instance Graphs for 3D Instance Segmentation with Single-point Supervision10 Aug 2022
- 2022Semantic Segmentation-Assisted Instance Feature Fusion for Multi-Level 3D Part Instance Segmentation9 Aug 2022
- 2022SuperLine3D: Self-supervised Line Segmentation and Description for LiDAR Point Cloud3 Aug 2022
50 newest of 86 All 86 (JSON)
- 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
- 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 299
- 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
- 2022SoftGroup for 3D Instance Segmentation on Point Cloudscited by 197
- 2021Hierarchical Aggregation for 3D Instance Segmentationcited by 126
- 2022PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Modelscited by 117
- 2021Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentationcited by 107
- 2021Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networkscited by 98
- 2021Exploiting Edge-Oriented Reasoning for 3D Point-based Scene Graph Analysiscited by 47
- 2021HVPR: Hybrid Voxel-Point Representation for Single-stage 3D Object Detectioncited by 47
- 2022Dynamic 3D Scene Analysis by Point Cloud Accumulationcited by 47
- 2023Mask-Attention-Free Transformer for 3D Instance Segmentationcited by 45
12 most cited of 84 All 84 (JSON)
- 2026PlantC2USeg: Cross-Scale Consistent Pre-Training for Few-Shot Unified Plant Point Cloud Segmentation2 Sep 2026
- 2026SegmentAnyTreeV2: Scaling Transformer-Based Tree Instance Segmentation Across Sensors, Platforms, and Forests6 Jun 2026
- 2026Joint 2D-3D Segmentation and Association in Street-level Imaging26 May 2026
- 2026SegVGGT: Joint 3D Reconstruction and Instance Segmentation from Multi-View Images20 Mar 2026
- 2025PinPoint3D: Fine-Grained 3D Part Segmentation from a Few Clicks30 Sep 2025
- 2025OVSeg3R: Learn Open-vocabulary Instance Segmentation from 2D via 3D Reconstruction28 Sep 2025
- 2025SegDINO3D: 3D Instance Segmentation Empowered by Both Image-Level and Object-Level 2D Features19 Sep 2025
- 2025ForestFormer3D: A Unified Framework for End-to-End Segmentation of Forest LiDAR 3D Point Clouds20 Jun 2025
- 2025SPPSFormer: High-quality Superpoint-based Transformer for Roof Plane Instance Segmentation from Point Clouds30 May 2025
- 2025Beyond the Final Layer: Hierarchical Query Fusion Transformer with Agent-Interpolation Initialization for 3D Instance Segmentation6 Feb 2025
- 2024ProtoSeg: A Prototype-Based Point Cloud Instance Segmentation Method3 Oct 2024
- 2024Rethinking Attention Module Design for Point Cloud Analysis27 Jul 2024
12 newest of 84 All 84 (JSON)