# Real-World On-Vehicle Evaluation of Embedding-Based Anomaly Detection

arXiv: https://arxiv.org/abs/2605.19744 · HTML: https://mlsanity.com/paper/2605.19744/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 17 references that are in the map.
- First posted on arXiv 19 May 2026.

## Builds on (most cited first, 13 of 13)

- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation](https://mlsanity.com/paper/2104.14812/) (2021)
- [Detecting the Unexpected via Image Resynthesis](https://mlsanity.com/paper/1904.07595/) (2019)
- [Anomaly Detection in Autonomous Driving: A Survey](https://mlsanity.com/paper/2204.07974/) (2022)
- [AnomalyDINO: Boosting Patch-based Few-shot Anomaly Detection with DINOv2](https://mlsanity.com/paper/2405.14529/) (2024)
- [Outlier detection by ensembling uncertainty with negative objectness](https://mlsanity.com/paper/2402.15374/) (2024)
- [Perception Datasets for Anomaly Detection in Autonomous Driving: A Survey](https://mlsanity.com/paper/2302.02790/) (2023)
- [AnoVox: A Benchmark for Multimodal Anomaly Detection in Autonomous Driving](https://mlsanity.com/paper/2405.07865/) (2024)
- [Dense Out-of-Distribution Detection by Robust Learning on Synthetic Negative Data](https://mlsanity.com/paper/2112.12833/) (2021)
- [CoCar NextGen: a Multi-Purpose Platform for Connected Autonomous Driving Research](https://mlsanity.com/paper/2404.17550/) (2024)
- [One Stack to Rule them All: To Drive Automated Vehicles, and Reach for the 4th level](https://mlsanity.com/paper/2404.02645/) (2024)
- [Vision Foundation Model Embedding-Based Semantic Anomaly Detection](https://mlsanity.com/paper/2505.07998/) (2025)
- [DINO-AD: Unsupervised Anomaly Detection with Frozen DINO-V3 Features](https://mlsanity.com/paper/2602.03870/) (2026)

## Abstract (arXiv)

Detecting anomalies in traffic scenes is crucial for ensuring safety in autonomous driving, yet collecting representative anomalous data remains challenging. Existing anomaly detection methods are highly specialized and rely on normality as defined by the abstract semantic Cityscapes classes, making it difficult to adapt to diverse real-world scenarios. We propose an adaptable real-time anomaly detection method that leverages foundation models in the form of pretrained vision transformer embeddings to detect deviations via nearest-neighbor similarity in the latent semantic feature space. Based on patch-wise processing, the algorithm produces dense anomaly masks, allowing for the localization of detected anomalies. The method robustly models normality through a single reference image. This formulation avoids explicit supervision and dataset-specific training, making it suitable for real-world deployment. We evaluate the method on standard benchmarks and on an automated vehicle in real-world scenarios. Despite its simplicity, the method achieves good performance on the Road Anomaly benchmark and demonstrates consistent qualitative behavior in practice, successfully highlighting semantically unusual objects in diverse scenes. These results suggest that simple, reference-based methods can provide useful anomaly signals under realistic operating conditions.