# Label-Free Model Failure Detection for Lidar-based Point Cloud Segmentation

arXiv: https://arxiv.org/abs/2407.14306 · HTML: https://mlsanity.com/paper/2407.14306/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 78 references that are in the map.
- First posted on arXiv 19 Jul 2024.

## Builds on (most cited first, 30 of 47)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving](https://mlsanity.com/paper/2003.03653/) (2020)
- [One Million Scenes for Autonomous Driving: ONCE Dataset](https://mlsanity.com/paper/2106.11037/) (2021)
- [Improving Adversarial Robustness via Promoting Ensemble Diversity](https://mlsanity.com/paper/1901.08846/) (2019)
- [Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning](https://mlsanity.com/paper/2003.12862/) (2020)
- [Are Labels Always Necessary for Classifier Accuracy Evaluation?](https://mlsanity.com/paper/2007.02915/) (2020)
- [CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Driving](https://mlsanity.com/paper/2203.07724/) (2022)
- [Just Go with the Flow: Self-Supervised Scene Flow Estimation](https://mlsanity.com/paper/1912.00497/) (2019)
- [Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift](https://mlsanity.com/paper/2206.13089/) (2022)
- [Anomaly Detection in Autonomous Driving: A Survey](https://mlsanity.com/paper/2204.07974/) (2022)
- [Identifying Unknown Instances for Autonomous Driving](https://mlsanity.com/paper/1910.11296/) (2019)
- [FlowStep3D: Model Unrolling for Self-Supervised Scene Flow Estimation](https://mlsanity.com/paper/2011.10147/) (2020)
- [Agree to Disagree: Diversity through Disagreement for Better Transferability](https://mlsanity.com/paper/2202.04414/) (2022)
- [Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data](https://mlsanity.com/paper/2105.08971/) (2021)
- ["Curse of rarity" for autonomous vehicles](https://mlsanity.com/paper/2207.02749/) (2022)
- [Revisiting Uncertainty-based Query Strategies for Active Learning with Transformers](https://mlsanity.com/paper/2107.05687/) (2021)
- [Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles](https://mlsanity.com/paper/2106.15728/) (2021)
- [Predicting Out-of-Distribution Error with the Projection Norm](https://mlsanity.com/paper/2202.05834/) (2022)
- [SODA10M: A Large-Scale 2D Self/Semi-Supervised Object Detection Dataset for Autonomous Driving](https://mlsanity.com/paper/2106.11118/) (2021)
- [Self-Supervised Learning of Non-Rigid Residual Flow and Ego-Motion](https://mlsanity.com/paper/2009.10467/) (2020)
- [Description of Corner Cases in Automated Driving: Goals and Challenges](https://mlsanity.com/paper/2109.09607/) (2021)
- [Unsupervised Universal Image Segmentation](https://mlsanity.com/paper/2312.17243/) (2023)
- [Outlier detection by ensembling uncertainty with negative objectness](https://mlsanity.com/paper/2402.15374/) (2024)
- [An Application-Driven Conceptualization of Corner Cases for Perception in Highly Automated Driving](https://mlsanity.com/paper/2103.03678/) (2021)
- [Open-set 3D Object Detection](https://mlsanity.com/paper/2112.01135/) (2021)

## Built on by (most cited first, 1 of 1)

- [Online Monitoring Framework for Automotive Time Series Data using JEPA Embeddings](https://mlsanity.com/paper/2602.09985/) (2026)

## Abstract (arXiv)

Autonomous vehicles drive millions of miles on the road each year. Under such circumstances, deployed machine learning models are prone to failure both in seemingly normal situations and in the presence of outliers. However, in the training phase, they are only evaluated on small validation and test sets, which are unable to reveal model failures due to their limited scenario coverage. While it is difficult and expensive to acquire large and representative labeled datasets for evaluation, large-scale unlabeled datasets are typically available. In this work, we introduce label-free model failure detection for lidar-based point cloud segmentation, taking advantage of the abundance of unlabeled data available. We leverage different data characteristics by training a supervised and self-supervised stream for the same task to detect failure modes. We perform a large-scale qualitative analysis and present LidarCODA, the first publicly available dataset with labeled anomalies in real-world lidar data, for an extensive quantitative analysis.