# Open-world Semantic Segmentation for LIDAR Point Clouds

arXiv: https://arxiv.org/abs/2207.01452 · HTML: https://mlsanity.com/paper/2207.01452/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 43 references that are in the map.
- First posted on arXiv 4 Jul 2022.

## Builds on (most cited first, 30 of 30)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [Scaling Out-of-Distribution Detection for Real-World Settings](https://mlsanity.com/paper/1911.11132/) (2019)
- [Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation](https://mlsanity.com/paper/2011.10033/) (2020)
- [Towards Open World Object Detection](https://mlsanity.com/paper/2103.02603/) (2021)
- [PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation](https://mlsanity.com/paper/2003.14032/) (2020)
- [SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud](https://mlsanity.com/paper/1809.08495/) (2018)
- [Towards Open World Recognition](https://mlsanity.com/paper/1412.5687/) (2014)
- [Incremental Learning of Object Detectors without Catastrophic Forgetting](https://mlsanity.com/paper/1708.06977/) (2017)
- [Modeling the Background for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/2002.00718/) (2020)
- [(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Network](https://mlsanity.com/paper/2102.04530/) (2021)
- [Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images](https://mlsanity.com/paper/1804.04488/) (2018)
- [Detecting the Unexpected via Image Resynthesis](https://mlsanity.com/paper/1904.07595/) (2019)
- [Learning Placeholders for Open-Set Recognition](https://mlsanity.com/paper/2103.15086/) (2021)
- [Encoder Based Lifelong Learning](https://mlsanity.com/paper/1704.01920/) (2017)
- [Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation](https://mlsanity.com/paper/2003.08440/) (2020)
- [Faster ILOD: Incremental Learning for Object Detectors based on Faster RCNN](https://mlsanity.com/paper/2003.03901/) (2020)
- [Learning Semantic Segmentation of Large-Scale Point Clouds with Random Sampling](https://mlsanity.com/paper/2107.02389/) (2021)
- [Deep Metric Learning for Open World Semantic Segmentation](https://mlsanity.com/paper/2108.04562/) (2021)
- [Energy-Based Open-World Uncertainty Modeling for Confidence Calibration](https://mlsanity.com/paper/2107.12628/) (2021)
- [PointSeg: Real-Time Semantic Segmentation Based on 3D LiDAR Point Cloud](https://mlsanity.com/paper/1807.06288/) (2018)

## Built on by (most cited first, 29 of 29)

- [Open-Vocabulary Point-Cloud Object Detection without 3D Annotation](https://mlsanity.com/paper/2304.00788/) (2023)
- [DeFlow: Decoder of Scene Flow Network in Autonomous Driving](https://mlsanity.com/paper/2401.16122/) (2024)
- [OpenSight: A Simple Open-Vocabulary Framework for LiDAR-Based Object Detection](https://mlsanity.com/paper/2312.08876/) (2023)
- [Novel Class Discovery for 3D Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2303.11610/) (2023)
- [A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application](https://mlsanity.com/paper/2310.14277/) (2023)
- [AnoVox: A Benchmark for Multimodal Anomaly Detection in Autonomous Driving](https://mlsanity.com/paper/2405.07865/) (2024)
- [3D Open-Vocabulary Panoptic Segmentation with 2D-3D Vision-Language Distillation](https://mlsanity.com/paper/2401.02402/) (2024)
- [An Empirical Study of Training State-of-the-Art LiDAR Segmentation Models](https://mlsanity.com/paper/2405.14870/) (2024)
- [PointCaM: Cut-and-Mix for Open-Set Point Cloud Learning](https://mlsanity.com/paper/2212.02011/) (2022)
- [Revisiting Out-of-Distribution Detection in LiDAR-based 3D Object Detection](https://mlsanity.com/paper/2404.15879/) (2024)
- [Segmenting Known Objects and Unseen Unknowns without Prior Knowledge](https://mlsanity.com/paper/2209.05407/) (2022)
- [Continual Learning for LiDAR Semantic Segmentation: Class-Incremental and Coarse-to-Fine strategies on Sparse Data](https://mlsanity.com/paper/2304.03980/) (2023)
- [Shape2Scene: 3D Scene Representation Learning Through Pre-training on Shape Data](https://mlsanity.com/paper/2407.10200/) (2024)
- [Lidar Panoptic Segmentation in an Open World](https://mlsanity.com/paper/2409.14273/) (2024)
- [Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation](https://mlsanity.com/paper/2505.16985/) (2025)
- [3D Indoor Instance Segmentation in an Open-World](https://mlsanity.com/paper/2309.14338/) (2023)
- [Open-Set LiDAR Panoptic Segmentation Guided by Uncertainty-Aware Learning](https://mlsanity.com/paper/2506.13265/) (2025)
- [Novel class discovery meets foundation models for 3D semantic segmentation](https://mlsanity.com/paper/2312.03782/) (2023)
- [PDF: A Probability-Driven Framework for Open World 3D Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2404.00979/) (2024)
- [LiDAR Remote Sensing Meets Weak Supervision: Concepts, Methods, and Perspectives](https://mlsanity.com/paper/2503.18384/) (2025)
- [Towards 3D Objectness Learning in an Open World](https://mlsanity.com/paper/2510.17686/) (2025)
- [Relative Energy Learning for LiDAR Out-of-Distribution Detection](https://mlsanity.com/paper/2511.06720/) (2025)
- [Point Cloud Video Anomaly Detection Based on Point Spatio-Temporal Auto-Encoder](https://mlsanity.com/paper/2306.04466/) (2023)
- [A Novel Decomposed Feature-Oriented Framework for Open-Set Semantic Segmentation on LiDAR Data](https://mlsanity.com/paper/2503.11097/) (2025)
- [Finding Outliers in a Haystack: Anomaly Detection for Large Pointcloud Scenes](https://mlsanity.com/paper/2508.17634/) (2025)
- [Depth-Only Open-Vocabulary 3D Semantic Segmentation For Privacy-Preserving Robotic Applications](https://mlsanity.com/paper/2607.00978/) (2026)
- [Open-world Point Cloud Semantic Segmentation: A Human-in-the-loop Framework](https://mlsanity.com/paper/2508.04962/) (2025)
- [UniFlow: Zero-Shot LiDAR Scene Flow for Autonomous Vehicles](https://mlsanity.com/paper/2511.18254/) (2025)
- [Neural Distribution Prior for LiDAR Out-of-Distribution Detection](https://mlsanity.com/paper/2604.09232/) (2026)

## Abstract (arXiv)

Current methods for LIDAR semantic segmentation are not robust enough for real-world applications, e.g., autonomous driving, since it is closed-set and static. The closed-set assumption makes the network only able to output labels of trained classes, even for objects never seen before, while a static network cannot update its knowledge base according to what it has seen. Therefore, in this work, we propose the open-world semantic segmentation task for LIDAR point clouds, which aims to 1) identify both old and novel classes using open-set semantic segmentation, and 2) gradually incorporate novel objects into the existing knowledge base using incremental learning without forgetting old classes. For this purpose, we propose a REdundAncy cLassifier (REAL) framework to provide a general architecture for both the open-set semantic segmentation and incremental learning problems. The experimental results show that REAL can simultaneously achieves state-of-the-art performance in the open-set semantic segmentation task on the SemanticKITTI and nuScenes datasets, and alleviate the catastrophic forgetting problem with a large margin during incremental learning.