# Clustering based Point Cloud Representation Learning for 3D Analysis

arXiv: https://arxiv.org/abs/2307.14605 · HTML: https://mlsanity.com/paper/2307.14605/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 85 of its 121 references that are in the map.
- First posted on arXiv 27 Jul 2023.

## Builds on (most cited first, 30 of 85)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds](https://mlsanity.com/paper/1911.11236/) (2019)
- [Self-labelling via simultaneous clustering and representation learning](https://mlsanity.com/paper/1911.05371/) (2019)
- [DeepGCNs: Can GCNs Go as Deep as CNNs?](https://mlsanity.com/paper/1904.03751/) (2019)
- [Dense Contrastive Learning for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2011.09157/) (2020)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution](https://mlsanity.com/paper/2007.16100/) (2020)
- [Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation](https://mlsanity.com/paper/2011.10033/) (2020)
- [Point Transformer V2: Grouped Vector Attention and Partition-based Pooling](https://mlsanity.com/paper/2210.05666/) (2022)

## Built on by (most cited first, 23 of 23)

- [A Survey of World Models for Autonomous Driving](https://mlsanity.com/paper/2501.11260/) (2025)
- [IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detection](https://mlsanity.com/paper/2403.15241/) (2024)
- [GroupContrast: Semantic-aware Self-supervised Representation Learning for 3D Understanding](https://mlsanity.com/paper/2403.09639/) (2024)
- [LOGICSEG: Parsing Visual Semantics with Neural Logic Learning and Reasoning](https://mlsanity.com/paper/2309.13556/) (2023)
- [3UR-LLM: An End-to-End Multimodal Large Language Model for 3D Scene Understanding](https://mlsanity.com/paper/2501.07819/) (2025)
- [LSK3DNet: Towards Effective and Efficient 3D Perception with Large Sparse Kernels](https://mlsanity.com/paper/2403.15173/) (2024)
- [Learning Clustering-based Prototypes for Compositional Zero-shot Learning](https://mlsanity.com/paper/2502.06501/) (2025)
- [Clustering Propagation for Universal Medical Image Segmentation](https://mlsanity.com/paper/2403.16646/) (2024)
- [Neural Clustering based Visual Representation Learning](https://mlsanity.com/paper/2403.17409/) (2024)
- [TrAME: Trajectory-Anchored Multi-View Editing for Text-Guided 3D Gaussian Splatting Manipulation](https://mlsanity.com/paper/2407.02034/) (2024)
- [Shape2Scene: 3D Scene Representation Learning Through Pre-training on Shape Data](https://mlsanity.com/paper/2407.10200/) (2024)
- [Clustering for Protein Representation Learning](https://mlsanity.com/paper/2404.00254/) (2024)
- [Entangled View-Epipolar Information Aggregation for Generalizable Neural Radiance Fields](https://mlsanity.com/paper/2311.11845/) (2023)
- [Subspace Prototype Guidance for Mitigating Class Imbalance in Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2408.10537/) (2024)
- [MsSVT++: Mixed-scale Sparse Voxel Transformer with Center Voting for 3D Object Detection](https://mlsanity.com/paper/2401.11718/) (2024)
- [Clustering-based Feature Representation Learning for Oracle Bone Inscriptions Detection](https://mlsanity.com/paper/2508.18641/) (2025)
- [Distribution Guidance Network for Weakly Supervised Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2410.08091/) (2024)
- [Generalized Robot 3D Vision-Language Model with Fast Rendering and Pre-Training Vision-Language Alignment](https://mlsanity.com/paper/2312.00663/) (2023)
- [Proto-FG3D: Prototype-based Interpretable Fine-Grained 3D Shape Classification](https://mlsanity.com/paper/2505.17666/) (2025)
- [ClusterStyle: Modeling Intra-Style Diversity with Prototypical Clustering for Stylized Motion Generation](https://mlsanity.com/paper/2512.02453/) (2025)
- [SPEGC: Continual Test-Time Adaptation via Semantic-Prompt-Enhanced Graph Clustering for Medical Image Segmentation](https://mlsanity.com/paper/2603.11492/) (2026)
- [4D-CS: Exploiting Cluster Prior for 4D Spatio-Temporal LiDAR Semantic Segmentation](https://mlsanity.com/paper/2501.02937/) (2025)
- [Hierarchical Direction Perception via Atomic Dot-Product Operators for Rotation-Invariant Point Clouds Learning](https://mlsanity.com/paper/2511.08240/) (2025)

## Abstract (arXiv)

Point cloud analysis (such as 3D segmentation and detection) is a challenging task, because of not only the irregular geometries of many millions of unordered points, but also the great variations caused by depth, viewpoint, occlusion, etc. Current studies put much focus on the adaption of neural networks to the complex geometries of point clouds, but are blind to a fundamental question: how to learn an appropriate point embedding space that is aware of both discriminative semantics and challenging variations? As a response, we propose a clustering based supervised learning scheme for point cloud analysis. Unlike current de-facto, scene-wise training paradigm, our algorithm conducts within-class clustering on the point embedding space for automatically discovering subclass patterns which are latent yet representative across scenes. The mined patterns are, in turn, used to repaint the embedding space, so as to respect the underlying distribution of the entire training dataset and improve the robustness to the variations. Our algorithm is principled and readily pluggable to modern point cloud segmentation networks during training, without extra overhead during testing. With various 3D network architectures (i.e., voxel-based, point-based, Transformer-based, automatically searched), our algorithm shows notable improvements on famous point cloud segmentation datasets (i.e.,2.0-2.6% on single-scan and 2.0-2.2% multi-scan of SemanticKITTI, 1.8-1.9% on S3DIS, in terms of mIoU). Our algorithm also demonstrates utility in 3D detection, showing 2.0-3.4% mAP gains on KITTI.