# Learning and Memorizing Representative Prototypes for 3D Point Cloud Semantic and Instance Segmentation

arXiv: https://arxiv.org/abs/2001.01349 · HTML: https://mlsanity.com/paper/2001.01349/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jan 2020.

## Built on by (most cited first, 26 of 26)

- [PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Models](https://mlsanity.com/paper/2212.01558/) (2022)
- [Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion](https://mlsanity.com/paper/2103.07074/) (2021)
- [PartSLIP++: Enhancing Low-Shot 3D Part Segmentation via Multi-View Instance Segmentation and Maximum Likelihood Estimation](https://mlsanity.com/paper/2312.03015/) (2023)
- [MaskGroup: Hierarchical Point Grouping and Masking for 3D Instance Segmentation](https://mlsanity.com/paper/2203.14662/) (2022)
- [The Neural Process Family: Survey, Applications and Perspectives](https://mlsanity.com/paper/2209.00517/) (2022)
- [Automated forest inventory: analysis of high-density airborne LiDAR point clouds with 3D deep learning](https://mlsanity.com/paper/2312.15084/) (2023)
- [Learning with Noisy Labels for Robust Point Cloud Segmentation](https://mlsanity.com/paper/2107.14230/) (2021)
- [PointInst3D: Segmenting 3D Instances by Points](https://mlsanity.com/paper/2204.11402/) (2022)
- [3D Part Segmentation via Geometric Aggregation of 2D Visual Features](https://mlsanity.com/paper/2412.04247/) (2024)
- [3D-QueryIS: A Query-based Framework for 3D Instance Segmentation](https://mlsanity.com/paper/2211.09375/) (2022)
- [MM-3DScene: 3D Scene Understanding by Customizing Masked Modeling with Informative-Preserved Reconstruction and Self-Distilled Consistency](https://mlsanity.com/paper/2212.09948/) (2022)
- [A Comprehensive Survey on Imbalanced Data Learning](https://mlsanity.com/paper/2502.08960/) (2025)
- [A Review of Panoptic Segmentation for Mobile Mapping Point Clouds](https://mlsanity.com/paper/2304.13980/) (2023)
- [Memory augment is All You Need for image restoration](https://mlsanity.com/paper/2309.01377/) (2023)
- [Towards 3D Scene Understanding by Referring Synthetic Models](https://mlsanity.com/paper/2203.10546/) (2022)
- [Deep Learning on 3D Semantic Segmentation: A Detailed Review](https://mlsanity.com/paper/2411.02104/) (2024)
- [PatchAlign3D: Local Feature Alignment for Dense 3D Shape Understanding](https://mlsanity.com/paper/2601.02457/) (2026)
- [Dynamic Convolution for 3D Point Cloud Instance Segmentation](https://mlsanity.com/paper/2107.08392/) (2021)
- [Geodesic-Former: a Geodesic-Guided Few-shot 3D Point Cloud Instance Segmenter](https://mlsanity.com/paper/2207.10859/) (2022)
- [Model2Scene: Learning 3D Scene Representation via Contrastive Language-CAD Models Pre-training](https://mlsanity.com/paper/2309.16956/) (2023)
- [Memory Regulation and Alignment toward Generalizer RGB-Infrared Person](https://mlsanity.com/paper/2109.08843/) (2021)
- [Improved Block Merging for 3D Point Cloud Instance Segmentation](https://mlsanity.com/paper/2407.06991/) (2024)
- [ProtoSeg: A Prototype-Based Point Cloud Instance Segmentation Method](https://mlsanity.com/paper/2410.02352/) (2024)
- [Joint prototype and coefficient prediction for 3D instance segmentation](https://mlsanity.com/paper/2407.06958/) (2024)
- [Towards scalable organ level 3D plant segmentation: Bridging the data algorithm computing gap](https://mlsanity.com/paper/2509.06329/) (2025)
- [PinPoint3D: Fine-Grained 3D Part Segmentation from a Few Clicks](https://mlsanity.com/paper/2509.25970/) (2025)

## Abstract (arXiv)

3D point cloud semantic and instance segmentation is crucial and fundamental for 3D scene understanding. Due to the complex structure, point sets are distributed off balance and diversely, which appears as both category imbalance and pattern imbalance. As a result, deep networks can easily forget the non-dominant cases during the learning process, resulting in unsatisfactory performance. Although re-weighting can reduce the influence of the well-classified examples, they cannot handle the non-dominant patterns during the dynamic training. In this paper, we propose a memory-augmented network to learn and memorize the representative prototypes that cover diverse samples universally. Specifically, a memory module is introduced to alleviate the forgetting issue by recording the patterns seen in mini-batch training. The learned memory items consistently reflect the interpretable and meaningful information for both dominant and non-dominant categories and cases. The distorted observations and rare cases can thus be augmented by retrieving the stored prototypes, leading to better performances and generalization. Exhaustive experiments on the benchmarks, i.e. S3DIS and ScanNetV2, reflect the superiority of our method on both effectiveness and efficiency. Not only the overall accuracy but also nondominant classes have improved substantially.