mlsanityData to 14 Oct 2026

Spotting the Unexpected (STU): A 3D LiDAR Dataset for Anomaly Segmentation in Autonomous Driving

arXiv 2505.02148 ↗First posted 4 May 2025cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

To operate safely, autonomous vehicles (AVs) need to detect and handle unexpected objects or anomalies on the road. While significant research exists for anomaly detection and segmentation in 2D, research progress in 3D is underexplored. Existing datasets lack high-quality multimodal data that are typically found in AVs. This paper presents a novel dataset for anomaly segmentation in driving scenarios. To the best of our knowledge, it is the first publicly available dataset focused on road anomaly segmentation with dense 3D semantic labeling, incorporating both LiDAR and camera data, as well as sequential information to enable anomaly detection across various ranges. This capability is critical for the safe navigation of autonomous vehicles. We adapted and evaluated several baseline models for 3D segmentation, highlighting the challenges of 3D anomaly detection in driving environments. Our dataset and evaluation code will be openly available, facilitating the testing and performance comparison of different approaches.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 45 of its 61 references that are in the map. First posted on arXiv 4 May 2025.

Builds on

45 of 61 references.

References · 45

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  2. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  6. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  7. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  8. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  9. 2018Panoptic Segmentationcited by 802
  10. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  11. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  12. 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
33 more
  1. 2021One Million Scenes for Autonomous Driving: ONCE Datasetcited by 218
  2. 2021SegmentMeIfYouCan: A Benchmark for Anomaly Segmentationcited by 150
  3. 2020Entropy Maximization and Meta Classification for Out-Of-Distribution Detection in Semantic Segmentationcited by 133
  4. 2022KISS-ICP: In Defense of Point-to-Point ICP -- Simple, Accurate, and Robust Registration If Done the Right Waycited by 132
  5. 2019The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentationcited by 122
  6. 2016Lost and Found: Detecting Small Road Hazards for Self-Driving Vehiclescited by 117
  7. 2022CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Drivingcited by 100
  8. 2022GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Modelscited by 87
  9. 2021Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentation on Complex Urban Driving Scenescited by 83
  10. 2021Standardized Max Logits: A Simple yet Effective Approach for Identifying Unexpected Road Obstacles in Urban-Scene Segmentationcited by 77
  11. 2019Identifying Unknown Instances for Autonomous Drivingcited by 74
  12. 2022RbA: Segmenting Unknown Regions Rejected by Allcited by 67
  13. 2019Simultaneous Semantic Segmentation and Outlier Detection in Presence of Domain Shiftcited by 54
  14. 2022DeepPrivacy2: Towards Realistic Full-Body Anonymizationcited by 51
  15. 2023Unmasking Anomalies in Road-Scene Segmentationcited by 45
  16. 2021SODA10M: A Large-Scale 2D Self/Semi-Supervised Object Detection Dataset for Autonomous Drivingcited by 43
  17. 2021Exemplar-Based Open-Set Panoptic Segmentation Networkcited by 38
  18. 2024Outlier detection by ensembling uncertainty with negative objectnesscited by 34
  19. 2020Dense open-set recognition with synthetic outliers generated by Real NVPcited by 29
  20. 2021Open-set 3D Object Detectioncited by 29
  21. 2023Mask4Former: Mask Transformer for 4D Panoptic Segmentationcited by 24
  22. 2022Two Video Data Sets for Tracking and Retrieval of Out of Distribution Objectscited by 19
  23. 2020LiDAR guided Small obstacle Segmentationcited by 17
  24. 2022Out-of-Distribution Detection for LiDAR-based 3D Object Detectioncited by 17
  25. 2023Perception Datasets for Anomaly Detection in Autonomous Driving: A Surveycited by 16
  26. 2019Automatic extrinsic calibration between a camera and a 3D Lidar using 3D point and plane correspondencescited by 15
  27. 2024AnoVox: A Benchmark for Multimodal Anomaly Detection in Autonomous Drivingcited by 14
  28. 2018MergeNet: A Deep Net Architecture for Small Obstacle Discoverycited by 11
  29. 2024Revisiting Out-of-Distribution Detection in LiDAR-based 3D Object Detectioncited by 11
  30. 2022Segmenting Known Objects and Unseen Unknowns without Prior Knowledgecited by 9
  31. 2024Lidar Panoptic Segmentation in an Open Worldcited by 8
  32. 2023LS-VOS: Identifying Outliers in 3D Object Detections Using Latent Space Virtual Outlier Synthesiscited by 7
  33. 2025Panoptic-CUDAL: Rural Australia Point Cloud Dataset in Rainy Conditionscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 6.

10 papers.

  1. 2025Relative Energy Learning for LiDAR Out-of-Distribution Detectioncited by 2
  2. 2025What Demands Attention in Urban Street Scenes? From Scene Understanding towards Road Safety: A Survey of Vision-driven Datasets and Studiescited by 1
  3. 2025Few-shot Human Action Anomaly Detection via a Unified Contrastive Learning Frameworkcited by 1
  4. 2025ClimaOoD: Improving Anomaly Segmentation via Physically Realistic Synthetic Datacited by 0
  5. 2026Neural Distribution Prior for LiDAR Out-of-Distribution Detectioncited by 0
  6. 2026Learning to Identify Out-of-Distribution Objects for 3D LiDAR Anomaly Segmentationcited by 0
  7. 2026LADBench: A Benchmark for Logical Fault Detection in Imagescited by 0
  8. 2026A Synthetic 3D Gear Dataset for Manufacturing Quality Inspection (MFGNet-Gear)cited by 0
  9. 2026NSF-HRPT: Neural Semantic Field meets Hierarchical Risk Perception Tree for Safety-Critical Scenario Assessmentcited by 0
  10. 2026TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perceptioncited by 0