# Labels Are Not Perfect: Inferring Spatial Uncertainty in Object Detection

arXiv: https://arxiv.org/abs/2012.12195 · HTML: https://mlsanity.com/paper/2012.12195/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Dec 2020.

## Built on by (most cited first, 18 of 18)

- [PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving](https://mlsanity.com/paper/2112.12610/) (2021)
- [Few-shot Object Counting and Detection](https://mlsanity.com/paper/2207.10988/) (2022)
- [A Simple and Efficient Multi-task Network for 3D Object Detection and Road Understanding](https://mlsanity.com/paper/2103.04056/) (2021)
- [Identifying Label Errors in Object Detection Datasets by Loss Inspection](https://mlsanity.com/paper/2303.06999/) (2023)
- [Bayesian Deep Learning for Affordance Segmentation in images](https://mlsanity.com/paper/2303.00871/) (2023)
- [Jaccard Metric Losses: Optimizing the Jaccard Index with Soft Labels](https://mlsanity.com/paper/2302.05666/) (2023)
- [Uncertainty-Aware Autonomous Vehicles: Predicting the Road Ahead](https://mlsanity.com/paper/2510.22680/) (2025)
- [Probabilistic Approach for Road-Users Detection](https://mlsanity.com/paper/2112.01360/) (2021)
- [Calibrating Perception Uncertainty for Autonomous Driving](https://mlsanity.com/paper/2211.04340/) (2022)
- [Run-time Monitoring of 3D Object Detection in Automated Driving Systems Using Early Layer Neural Activation Patterns](https://mlsanity.com/paper/2404.07685/) (2024)
- [Drawing the Same Bounding Box Twice? Coping Noisy Annotations in Object Detection with Repeated Labels](https://mlsanity.com/paper/2309.09742/) (2023)
- [L2M-Reg: Building-level Uncertainty-aware Registration of Outdoor LiDAR Point Clouds and Semantic 3D City Models](https://mlsanity.com/paper/2509.16832/) (2025)
- [Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model](https://mlsanity.com/paper/2111.11055/) (2021)
- [MEDL-U: Uncertainty-aware 3D Automatic Annotation based on Evidential Deep Learning](https://mlsanity.com/paper/2309.09599/) (2023)
- [Aligning Object Detector Bounding Boxes with Human Preference](https://mlsanity.com/paper/2408.10844/) (2024)
- [A Method for the Runtime Validation of AI-based Environment Perception in Automated Driving System](https://mlsanity.com/paper/2412.16762/) (2024)
- [Perception Characteristics Distance: Measuring Stability and Robustness of Perception System in Dynamic Conditions under a Certain Decision Rule](https://mlsanity.com/paper/2506.09217/) (2025)
- [Quantifying Annotation Agreement in Object Detection with Optimal Transport](https://mlsanity.com/paper/2610.19912/) (2026)

## Abstract (arXiv)

The availability of many real-world driving datasets is a key reason behind the recent progress of object detection algorithms in autonomous driving. However, there exist ambiguity or even failures in object labels due to error-prone annotation process or sensor observation noise. Current public object detection datasets only provide deterministic object labels without considering their inherent uncertainty, as does the common training process or evaluation metrics for object detectors. As a result, an in-depth evaluation among different object detection methods remains challenging, and the training process of object detectors is sub-optimal, especially in probabilistic object detection. In this work, we infer the uncertainty in bounding box labels from LiDAR point clouds based on a generative model, and define a new representation of the probabilistic bounding box through a spatial uncertainty distribution. Comprehensive experiments show that the proposed model reflects complex environmental noises in LiDAR perception and the label quality. Furthermore, we propose Jaccard IoU (JIoU) as a new evaluation metric that extends IoU by incorporating label uncertainty. We conduct an in-depth comparison among several LiDAR-based object detectors using the JIoU metric. Finally, we incorporate the proposed label uncertainty in a loss function to train a probabilistic object detector and to improve its detection accuracy. We verify our proposed methods on two public datasets (KITTI, Waymo), as well as on simulation data. Code is released at https://bit.ly/2W534yo.