# CUS3D :CLIP-based Unsupervised 3D Segmentation via Object-level Denoise

arXiv: https://arxiv.org/abs/2409.13982 · HTML: https://mlsanity.com/paper/2409.13982/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 22 references that are in the map.
- First posted on arXiv 21 Sep 2024.

## Builds on (most cited first, 4 of 4)

- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [CLIP-FO3D: Learning Free Open-world 3D Scene Representations from 2D Dense CLIP](https://mlsanity.com/paper/2303.04748/) (2023)
- [U3DS3: Unsupervised 3D Semantic Scene Segmentation](https://mlsanity.com/paper/2311.06018/) (2023)
- [Zero-shot point cloud segmentation by transferring geometric primitives](https://mlsanity.com/paper/2210.09923/) (2022)

## Abstract (arXiv)

To ease the difficulty of acquiring annotation labels in 3D data, a common method is using unsupervised and open-vocabulary semantic segmentation, which leverage 2D CLIP semantic knowledge. In this paper, unlike previous research that ignores the ``noise'' raised during feature projection from 2D to 3D, we propose a novel distillation learning framework named CUS3D. In our approach, an object-level denosing projection module is designed to screen out the ``noise'' and ensure more accurate 3D feature. Based on the obtained features, a multimodal distillation learning module is designed to align the 3D feature with CLIP semantic feature space with object-centered constrains to achieve advanced unsupervised semantic segmentation. We conduct comprehensive experiments in both unsupervised and open-vocabulary segmentation, and the results consistently showcase the superiority of our model in achieving advanced unsupervised segmentation results and its effectiveness in open-vocabulary segmentation.