# JSNet: Joint Instance and Semantic Segmentation of 3D Point Clouds

arXiv: https://arxiv.org/abs/1912.09654 · HTML: https://mlsanity.com/paper/1912.09654/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Dec 2019.

## Built on by (most cited first, 27 of 27)

- [SSPC-Net: Semi-supervised Semantic 3D Point Cloud Segmentation Network](https://mlsanity.com/paper/2104.07861/) (2021)
- [Exploiting Edge-Oriented Reasoning for 3D Point-based Scene Graph Analysis](https://mlsanity.com/paper/2103.05558/) (2021)
- [LIF-Seg: LiDAR and Camera Image Fusion for 3D LiDAR Semantic Segmentation](https://mlsanity.com/paper/2108.07511/) (2021)
- [UrbanBIS: a Large-scale Benchmark for Fine-grained Urban Building Instance Segmentation](https://mlsanity.com/paper/2305.02627/) (2023)
- [Boundary-Aware Geometric Encoding for Semantic Segmentation of Point Clouds](https://mlsanity.com/paper/2101.02381/) (2021)
- [Omni-supervised Point Cloud Segmentation via Gradual Receptive Field Component Reasoning](https://mlsanity.com/paper/2105.10203/) (2021)
- [PointInst3D: Segmenting 3D Instances by Points](https://mlsanity.com/paper/2204.11402/) (2022)
- [A comprehensive overview of deep learning techniques for 3D point cloud classification and semantic segmentation](https://mlsanity.com/paper/2405.11903/) (2024)
- [3D Instance Segmentation of MVS Buildings](https://mlsanity.com/paper/2112.09902/) (2021)
- [LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network](https://mlsanity.com/paper/2203.07186/) (2022)
- [Learning Scene Dynamics from Point Cloud Sequences](https://mlsanity.com/paper/2111.08755/) (2021)
- [Grounding Stylistic Domain Generalization with Quantitative Domain Shift Measures and Synthetic Scene Images](https://mlsanity.com/paper/2405.15961/) (2024)
- [Learning from Mistakes: Self-Regularizing Hierarchical Representations in Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2301.11145/) (2023)
- [PIG-Net: Inception based Deep Learning Architecture for 3D Point Cloud Segmentation](https://mlsanity.com/paper/2101.11987/) (2021)
- [Advancements in Point Cloud-Based 3D Defect Detection and Classification for Industrial Systems: A Comprehensive Survey](https://mlsanity.com/paper/2402.12923/) (2024)
- [Multi Point-Voxel Convolution (MPVConv) for Deep Learning on Point Clouds](https://mlsanity.com/paper/2107.13152/) (2021)
- [Semantic Segmentation-Assisted Instance Feature Fusion for Multi-Level 3D Part Instance Segmentation](https://mlsanity.com/paper/2208.04766/) (2022)
- [Improved Block Merging for 3D Point Cloud Instance Segmentation](https://mlsanity.com/paper/2407.06991/) (2024)
- [ProtoSeg: A Prototype-Based Point Cloud Instance Segmentation Method](https://mlsanity.com/paper/2410.02352/) (2024)
- [TriangleNet: Edge Prior Augmented Network for Semantic Segmentation through Cross-Task Consistency](https://mlsanity.com/paper/2210.05152/) (2022)
- [3DFusion, A real-time 3D object reconstruction pipeline based on streamed instance segmented data](https://mlsanity.com/paper/2311.06659/) (2023)
- [EipFormer: Emphasizing Instance Positions in 3D Instance Segmentation](https://mlsanity.com/paper/2312.05602/) (2023)
- [Joint prototype and coefficient prediction for 3D instance segmentation](https://mlsanity.com/paper/2407.06958/) (2024)
- [Multi Voxel-Point Neurons Convolution (MVPConv) for Fast and Accurate 3D Deep Learning](https://mlsanity.com/paper/2104.14834/) (2021)
- [JSMNet Improving Indoor Point Cloud Semantic and Instance Segmentation through Self-Attention and Multiscale](https://mlsanity.com/paper/2309.07425/) (2023)
- [Fine-grained Metrics for Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2407.21289/) (2024)
- [Methods for the Segmentation of Reticular Structures Using 3D LiDAR Data: A Comparative Evaluation](https://mlsanity.com/paper/2507.20589/) (2025)

## Abstract (arXiv)

In this paper, we propose a novel joint instance and semantic segmentation approach, which is called JSNet, in order to address the instance and semantic segmentation of 3D point clouds simultaneously. Firstly, we build an effective backbone network to extract robust features from the raw point clouds. Secondly, to obtain more discriminative features, a point cloud feature fusion module is proposed to fuse the different layer features of the backbone network. Furthermore, a joint instance semantic segmentation module is developed to transform semantic features into instance embedding space, and then the transformed features are further fused with instance features to facilitate instance segmentation. Meanwhile, this module also aggregates instance features into semantic feature space to promote semantic segmentation. Finally, the instance predictions are generated by applying a simple mean-shift clustering on instance embeddings. As a result, we evaluate the proposed JSNet on a large-scale 3D indoor point cloud dataset S3DIS and a part dataset ShapeNet, and compare it with existing approaches. Experimental results demonstrate our approach outperforms the state-of-the-art method in 3D instance segmentation with a significant improvement in 3D semantic prediction and our method is also beneficial for part segmentation. The source code for this work is available at https://github.com/dlinzhao/JSNet.