mlsanityData to 7 Oct 2026

Calib3D: Calibrating Model Preferences for Reliable 3D Scene Understanding

arXiv 2403.17010 ↗First posted 25 Mar 2024cs.CV cs.LG cs.ROcited by 11

Not on the map. Why?

Abstract (arXiv)

Safety-critical 3D scene understanding tasks necessitate not only accurate but also confident predictions from 3D perception models. This study introduces Calib3D, a pioneering effort to benchmark and scrutinize the reliability of 3D scene understanding models from an uncertainty estimation viewpoint. We comprehensively evaluate 28 state-of-the-art models across 10 diverse 3D datasets, uncovering insightful phenomena that cope with both the aleatoric and epistemic uncertainties in 3D scene understanding. We discover that despite achieving impressive levels of accuracy, existing models frequently fail to provide reliable uncertainty estimates -- a pitfall that critically undermines their applicability in safety-sensitive contexts. Through extensive analysis of key factors such as network capacity, LiDAR representations, rasterization resolutions, and 3D data augmentation techniques, we correlate these aspects directly with the model calibration efficacy. Furthermore, we introduce DeptS, a novel depth-aware scaling approach aimed at enhancing 3D model calibration. Extensive experiments across a wide range of configurations validate the superiority of our method. We hope this work could serve as a cornerstone for fostering reliable 3D scene understanding. Code and benchmark toolkit are publicly available.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 111 of its 146 references that are in the map. First posted on arXiv 25 Mar 2024.

Builds on

111 of 146 references.

References · 111

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  7. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  8. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  9. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  10. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  11. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  12. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
38 more
  1. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  2. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
  3. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  4. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
  5. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  6. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 728
  7. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  8. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 584
  9. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
  10. 2019Measuring Calibration in Deep Learningcited by 443
  11. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 422
  12. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 388
  13. 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 384
  14. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  15. 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 359
  16. 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 351
  17. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 274
  18. 2020SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Drivingcited by 226
  19. 2021PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Cloudscited by 211
  20. 2020Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learningcited by 208
  21. 2019Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Visioncited by 182
  22. 2020On the uncertainty of self-supervised monocular depth estimationcited by 182
  23. 2023CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIPcited by 178
  24. 2020Training independent subnetworks for robust predictioncited by 177
  25. 2021RPVNet: A Deep and Efficient Range-Point-Voxel Fusion Network for LiDAR Point Cloud Segmentationcited by 173
  26. 2020SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentationcited by 170
  27. 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 165
  28. 2021(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Networkcited by 164
  29. 2019Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentationcited by 162
  30. 20222DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Cloudscited by 157
  31. 2018Evaluating Bayesian Deep Learning Methods for Semantic Segmentationcited by 143
  32. 2020Uncertainty Quantification and Deep Ensemblescited by 138
  33. 2018Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flowcited by 135
  34. 2019xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentationcited by 129
  35. 2023Robo3D: Towards Robust and Reliable 3D Perception against Corruptionscited by 128
  36. 2023Rethinking Range View Representation for LiDAR Segmentationcited by 126
  37. 2020SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instancescited by 123
  38. 2023Segment Any Point Cloud Sequences by Distilling Vision Foundation Modelscited by 123

50 most cited of 111 All 111 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 5; 2026: 2.

11 papers.

  1. 2024Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous Drivingcited by 53
  2. 2024Multi-Space Alignments Towards Universal LiDAR Segmentationcited by 24
  3. 2025Perspective-Invariant 3D Object Detectioncited by 16
  4. 2024An Empirical Study of Training State-of-the-Art LiDAR Segmentation Modelscited by 12
  5. 2025SPIRAL: Semantic-Aware Progressive LiDAR Scene Generation and Understandingcited by 10
  6. 2025Beyond One Shot, Beyond One Perspective: Cross-View and Long-Horizon Distillation for Better LiDAR Representationscited by 7
  7. 2024ReliOcc: Towards Reliable Semantic Occupancy Prediction via Uncertainty Learningcited by 6
  8. 2025U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequencescited by 5
  9. 2025Zero-Shot 3D Visual Grounding from Vision-Language Modelscited by 2
  10. 2026Query2Uncertainty: Robust Uncertainty Quantification and Calibration for 3D Object Detection under Distribution Shiftcited by 1
  11. 2026Learning to Identify Out-of-Distribution Objects for 3D LiDAR Anomaly Segmentationcited by 0