mlsanityData to 14 Oct 2026

Have We Ever Encountered This Before? Retrieving Out-of-Distribution Road Obstacles from Driving Scenes

arXiv 2309.04302 ↗First posted 8 Sep 2023cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

In the life cycle of highly automated systems operating in an open and dynamic environment, the ability to adjust to emerging challenges is crucial. For systems integrating data-driven AI-based components, rapid responses to deployment issues require fast access to related data for testing and reconfiguration. In the context of automated driving, this especially applies to road obstacles that were not included in the training data, commonly referred to as out-of-distribution (OoD) road obstacles. Given the availability of large uncurated recordings of driving scenes, a pragmatic approach is to query a database to retrieve similar scenarios featuring the same safety concerns due to OoD road obstacles. In this work, we extend beyond identifying OoD road obstacles in video streams and offer a comprehensive approach to extract sequences of OoD road obstacles using text queries, thereby proposing a way of curating a collection of OoD data for subsequent analysis. Our proposed method leverages the recent advances in OoD segmentation and multi-modal foundation models to identify and efficiently extract safety-relevant scenes from unlabeled videos. We present a first approach for the novel task of text-based OoD object retrieval, which addresses the question ''Have we ever encountered this before?''.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 52 of its 69 references that are in the map. First posted on arXiv 8 Sep 2023.

Builds on

52 of 69 references.

References · 52

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2016Densely Connected Convolutional Networkscited by 6,720
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  7. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  8. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  9. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  10. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  11. 2015Fast R-CNNcited by 2,933
  12. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
38 more
  1. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  2. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  3. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  4. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  5. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 910
  6. 2016Simple Online and Realtime Trackingcited by 868
  7. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 651
  8. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
  9. 2020Tracking Objects as Pointscited by 483
  10. 2019Tracking without bells and whistlescited by 382
  11. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  12. 2019MOTS: Multi-Object Tracking and Segmentationcited by 223
  13. 2019Learning a Neural Solver for Multiple Object Trackingcited by 198
  14. 2021SegmentMeIfYouCan: A Benchmark for Anomaly Segmentationcited by 150
  15. 2020Entropy Maximization and Meta Classification for Out-Of-Distribution Detection in Semantic Segmentationcited by 133
  16. 2019Classifying, Segmenting, and Tracking Object Instances in Video with Mask Propagationcited by 125
  17. 2015Natural Language Object Retrievalcited by 124
  18. 2016Lost and Found: Detecting Small Road Hazards for Self-Driving Vehiclescited by 117
  19. 2019The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentationcited by 97
  20. 2016Learning by tracking: Siamese CNN for robust target associationcited by 79
  21. 2021Everything at Once -- Multi-modal Fusion Transformer for Video Retrievalcited by 79
  22. 2018Lightweight Probabilistic Deep Networkscited by 77
  23. 2021Standardized Max Logits: A Simple yet Effective Approach for Identifying Unexpected Road Obstacles in Urban-Scene Segmentationcited by 77
  24. 2022Disentangled Representation Learning for Text-Video Retrievalcited by 77
  25. 2020A Decade Survey of Content Based Image Retrieval using Deep Learningcited by 72
  26. 2021Deep Learning for Instance Retrieval: A Surveycited by 70
  27. 2016T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videoscited by 67
  28. 2022RbA: Segmenting Unknown Regions Rejected by Allcited by 67
  29. 2018Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilitiescited by 54
  30. 2021Opening up Open-World Trackingcited by 50
  31. 2023On Advantages of Mask-level Recognition for Outlier-aware Segmentationcited by 42
  32. 2020Detecting Road Obstacles by Erasing Themcited by 29
  33. 2023Survey on Unsupervised Domain Adaptation for Semantic Segmentation for Visual Perception in Automated Drivingcited by 29
  34. 2017Object Detection in Videos with Tubelet Proposal Networkscited by 24
  35. 2019Time-Dynamic Estimates of the Reliability of Deep Semantic Segmentation Networkscited by 20
  36. 2020Detection and Retrieval of Out-of-Distribution Objects in Semantic Segmentationcited by 20
  37. 2022Two Video Data Sets for Tracking and Retrieval of Out of Distribution Objectscited by 19
  38. 2019Uncertainty Measures and Prediction Quality Rating for the Semantic Segmentation of Nested Multi Resolution Street Scene Imagescited by 18

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 2.

9 papers.

  1. 2025Out-of-Distribution Segmentation in Autonomous Driving: Problems and State of the Artcited by 13
  2. 2024A Likelihood Ratio-Based Approach to Segmenting Unknown Objectscited by 12
  3. 2024Segment-Level Road Obstacle Detection Using Visual Foundation Model Priors and Likelihood Ratioscited by 3
  4. 2025Adaptive Neural Networks for Intelligent Data-Driven Developmentcited by 3
  5. 2025Open-set Anomaly Segmentation in Complex Scenarioscited by 1
  6. 2025From Pixel to Mask: A Survey of Out-of-Distribution Segmentationcited by 1
  7. 2024An Augmentation-based Model Re-adaptation Framework for Robust Image Segmentationcited by 0
  8. 2026CURE-OOD: Benchmarking Out-of-Distribution Detection for Survival Predictioncited by 0
  9. 2026Combining Foundation Model Confidence and Monocular Depth for Training-Free Out-of-Distribution Segmentationcited by 0