# Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming

arXiv: https://arxiv.org/abs/1907.07484 · HTML: https://mlsanity.com/paper/1907.07484/

## Key facts

- Cited by 336 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2019.

## Built on by (most cited first, 30 of 336)

- [SWAD: Domain Generalization by Seeking Flat Minima](https://mlsanity.com/paper/2102.08604/) (2021)
- [Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision](https://mlsanity.com/paper/2309.14181/) (2023)
- [DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2203.08195/) (2022)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [Partial success in closing the gap between human and machine vision](https://mlsanity.com/paper/2106.07411/) (2021)
- [Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models](https://mlsanity.com/paper/2311.06783/) (2023)
- [SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation](https://mlsanity.com/paper/2206.08367/) (2022)
- [Domain Generalization by Mutual-Information Regularization with Pre-trained Models](https://mlsanity.com/paper/2203.10789/) (2022)
- [The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization](https://mlsanity.com/paper/2112.00463/) (2021)
- [Robo3D: Towards Robust and Reliable 3D Perception against Corruptions](https://mlsanity.com/paper/2303.17597/) (2023)
- [3D Common Corruptions and Data Augmentation](https://mlsanity.com/paper/2203.01441/) (2022)
- [Benchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Driving](https://mlsanity.com/paper/2303.11040/) (2023)
- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.03721/) (2021)
- [Towards Open-ended Visual Quality Comparison](https://mlsanity.com/paper/2402.16641/) (2024)
- [Stronger, Fewer, & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2312.04265/) (2023)
- [Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2108.11249/) (2021)
- [Benchmarking Robustness of 3D Point Cloud Recognition Against Common Corruptions](https://mlsanity.com/paper/2201.12296/) (2022)
- [Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys](https://mlsanity.com/paper/2303.17220/) (2023)
- [Wavelet Convolutions for Large Receptive Fields](https://mlsanity.com/paper/2407.05848/) (2024)
- [Test-Time Adaptation to Distribution Shift by Confidence Maximization and Input Transformation](https://mlsanity.com/paper/2106.14999/) (2021)
- [Probable Domain Generalization via Quantile Risk Minimization](https://mlsanity.com/paper/2207.09944/) (2022)
- [Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy](https://mlsanity.com/paper/2103.03014/) (2021)
- [On the Practicality of Deterministic Epistemic Uncertainty](https://mlsanity.com/paper/2107.00649/) (2021)
- [Visual Representation Learning Does Not Generalize Strongly Within the Same Domain](https://mlsanity.com/paper/2107.08221/) (2021)
- [Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2304.10410/) (2023)
- [Learning Domain Adaptive Object Detection with Probabilistic Teacher](https://mlsanity.com/paper/2206.06293/) (2022)
- [RobustNav: Towards Benchmarking Robustness in Embodied Navigation](https://mlsanity.com/paper/2106.04531/) (2021)
- [An Efficient Domain-Incremental Learning Approach to Drive in All Weather Conditions](https://mlsanity.com/paper/2204.08817/) (2022)
- [Meaningfully Debugging Model Mistakes using Conceptual Counterfactual Explanations](https://mlsanity.com/paper/2106.12723/) (2021)

## Abstract (arXiv)

The ability to detect objects regardless of image distortions or weather conditions is crucial for real-world applications of deep learning like autonomous driving. We here provide an easy-to-use benchmark to assess how object detection models perform when image quality degrades. The three resulting benchmark datasets, termed Pascal-C, Coco-C and Cityscapes-C, contain a large variety of image corruptions. We show that a range of standard object detection models suffer a severe performance loss on corrupted images (down to 30--60% of the original performance). However, a simple data augmentation trick---stylizing the training images---leads to a substantial increase in robustness across corruption type, severity and dataset. We envision our comprehensive benchmark to track future progress towards building robust object detection models. Benchmark, code and data are publicly available.