mlsanityData to 7 Oct 2026

Multimodal Detection of Unknown Objects on Roads for Autonomous Driving

arXiv 2205.01414 ↗First posted 3 May 2022cs.CV cs.LG cs.ROcited by 4

Not on the map. Why?

Abstract (arXiv)

Tremendous progress in deep learning over the last years has led towards a future with autonomous vehicles on our roads. Nevertheless, the performance of their perception systems is strongly dependent on the quality of the utilized training data. As these usually only cover a fraction of all object classes an autonomous driving system will face, such systems struggle with handling the unexpected. In order to safely operate on public roads, the identification of objects from unknown classes remains a crucial task. In this paper, we propose a novel pipeline to detect unknown objects. Instead of focusing on a single sensor modality, we make use of lidar and camera data by combining state-of-the art detection models in a sequential manner. We evaluate our approach on the Waymo Open Perception Dataset and point out current research gaps in anomaly detection.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 42 of its 64 references that are in the map. First posted on arXiv 3 May 2022.

Builds on

42 of 64 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2017On Calibration of Modern Neural Networkscited by 4,860
  8. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  9. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  10. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  11. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  12. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
30 more
  1. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  2. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  3. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  4. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  5. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  6. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  7. 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
  8. 2021Towards Open World Object Detectioncited by 297
  9. 2015Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understandingcited by 218
  10. 2021One Million Scenes for Autonomous Driving: ONCE Datasetcited by 218
  11. 2019High-Resolution Representations for Labeling Pixels and Regionscited by 211
  12. 2018IDD: A Dataset for Exploring Problems of Autonomous Navigation in Unconstrained Environmentscited by 163
  13. 2019Detecting the Unexpected via Image Resynthesiscited by 137
  14. 2020Entropy Maximization and Meta Classification for Out-Of-Distribution Detection in Semantic Segmentationcited by 133
  15. 2019The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentationcited by 122
  16. 2021MSeg: A Composite Dataset for Multi-domain Semantic Segmentationcited by 122
  17. 2021Pixel-wise Anomaly Detection in Complex Driving Scenescited by 109
  18. 2022CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Drivingcited by 100
  19. 2022Anomaly Detection in Autonomous Driving: A Surveycited by 87
  20. 2021Standardized Max Logits: A Simple yet Effective Approach for Identifying Unexpected Road Obstacles in Urban-Scene Segmentationcited by 77
  21. 2019Identifying Unknown Instances for Autonomous Drivingcited by 73
  22. 2018Efficient Uncertainty Estimation for Semantic Segmentation in Videoscited by 46
  23. 2020Out-of-Distribution Detection for Automotive Perceptioncited by 37
  24. 2020Real-time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection for Road-driving Imagescited by 36
  25. 2021Description of Corner Cases in Automated Driving: Goals and Challengescited by 35
  26. 2021Open-set 3D Object Detectioncited by 29
  27. 2018MergeNet: A Deep Net Architecture for Small Obstacle Discoverycited by 11
  28. 2022Ad-datasets: a meta-collection of data sets for autonomous drivingcited by 11
  29. 2020InsClustering: Instantly Clustering LiDAR Range Measures for Autonomous Vehiclecited by 3
  30. 2021Taxonomy and Survey on Remote Human Input Systems for Driving Automation Systemscited by 3

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 1.

4 papers.

  1. 2023Perception Datasets for Anomaly Detection in Autonomous Driving: A Surveycited by 16
  2. 2025Mcity Data Engine: Iterative Model Improvement Through Open-Vocabulary Data Selectioncited by 3
  3. 2024Fisheye Camera and Ultrasonic Sensor Fusion For Near-Field Obstacle Perception in Bird's-Eye-Viewcited by 2
  4. 2023Attention Based Feature Fusion For Multi-Agent Collaborative Perceptioncited by 1