# Language-Level Semantics Conditioned 3D Point Cloud Segmentation arXiv: https://arxiv.org/abs/2107.00430 ยท HTML: https://mlsanity.com/paper/2107.00430/ ## Key facts - Cited by 7 papers in the map (data through 14 Oct 2026). - Builds on 46 of its 68 references that are in the map. - First posted on arXiv 1 Jul 2021. ## Builds on (most cited first, 30 of 46) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016) - [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017) - [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017) - [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013) - [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013) - [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017) - [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018) - [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015) - [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019) - [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019) - [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018) - [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017) - [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017) - [RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds](https://mlsanity.com/paper/1911.11236/) (2019) - [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017) - [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019) - [Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs](https://mlsanity.com/paper/1711.09869/) (2017) - [Relation-Shape Convolutional Neural Network for Point Cloud Analysis](https://mlsanity.com/paper/1904.07601/) (2019) - [PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation](https://mlsanity.com/paper/2003.14032/) (2020) - [SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters](https://mlsanity.com/paper/1803.11527/) (2018) - [Feature Generating Networks for Zero-Shot Learning](https://mlsanity.com/paper/1712.00981/) (2017) - [Zero-Shot Semantic Segmentation](https://mlsanity.com/paper/1906.00817/) (2019) - [SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud](https://mlsanity.com/paper/1710.07368/) (2017) - [Deep Parametric Continuous Convolutional Neural Networks](https://mlsanity.com/paper/2101.06742/) (2021) - [Modeling Point Clouds with Self-Attention and Gumbel Subset Sampling](https://mlsanity.com/paper/1904.03375/) (2019) - [PointMixup: Augmentation for Point Clouds](https://mlsanity.com/paper/2008.06374/) (2020) - [Context-aware Feature Generation for Zero-shot Semantic Segmentation](https://mlsanity.com/paper/2008.06893/) (2020) - [Multiresolution Tree Networks for 3D Point Cloud Processing](https://mlsanity.com/paper/1807.03520/) (2018) - [A Closer Look at Local Aggregation Operators in Point Cloud Analysis](https://mlsanity.com/paper/2007.01294/) (2020) ## Built on by (most cited first, 7 of 7) - [OpenScene: 3D Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2211.15654/) (2022) - [A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Future](https://mlsanity.com/paper/2307.09220/) (2023) - [CoDA: Collaborative Novel Box Discovery and Cross-modal Alignment for Open-vocabulary 3D Object Detection](https://mlsanity.com/paper/2310.02960/) (2023) - [OpenSight: A Simple Open-Vocabulary Framework for LiDAR-Based Object Detection](https://mlsanity.com/paper/2312.08876/) (2023) - [See More and Know More: Zero-shot Point Cloud Segmentation via Multi-modal Visual Data](https://mlsanity.com/paper/2307.10782/) (2023) - [Collaborative Novel Object Discovery and Box-Guided Cross-Modal Alignment for Open-Vocabulary 3D Object Detection](https://mlsanity.com/paper/2406.00830/) (2024) - [Open-Vocabulary 3D Semantic Segmentation with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2407.13642/) (2024) ## Abstract (arXiv) In this work, a language-level Semantics Conditioned framework for 3D Point cloud segmentation, called SeCondPoint, is proposed, where language-level semantics are introduced to condition the modeling of point feature distribution as well as the pseudo-feature generation, and a feature-geometry-based mixup approach is further proposed to facilitate the distribution learning. To our knowledge, this is the first attempt in literature to introduce language-level semantics to the 3D point cloud segmentation task. Since a large number of point features could be generated from the learned distribution thanks to the semantics conditioned modeling, any existing segmentation network could be embedded into the proposed framework to boost its performance. In addition, the proposed framework has the inherent advantage of dealing with novel classes, which seems an impossible feat for the current segmentation networks. Extensive experimental results on two public datasets demonstrate that three typical segmentation networks could achieve significant improvements over their original performances after enhancement by the proposed framework in the conventional 3D segmentation task. Two benchmarks are also introduced for a newly introduced zero-shot 3D segmentation task, and the results also validate the proposed framework.