# Using the Overlapping Score to Improve Corruption Benchmarks

arXiv: https://arxiv.org/abs/2105.12357 · HTML: https://mlsanity.com/paper/2105.12357/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 20 references that are in the map.
- First posted on arXiv 26 May 2021.

## Builds on (most cited first, 15 of 15)

- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Measuring Robustness to Natural Distribution Shifts in Image Classification](https://mlsanity.com/paper/2007.00644/) (2020)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [Noise or Signal: The Role of Image Backgrounds in Object Recognition](https://mlsanity.com/paper/2006.09994/) (2020)
- [Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming](https://mlsanity.com/paper/1907.07484/) (2019)
- [A Fourier Perspective on Model Robustness in Computer Vision](https://mlsanity.com/paper/1906.08988/) (2019)
- [Adversarial Training and Robustness for Multiple Perturbations](https://mlsanity.com/paper/1904.13000/) (2019)
- [Examining the Impact of Blur on Recognition by Convolutional Networks](https://mlsanity.com/paper/1611.05760/) (2016)
- [CURE-TSR: Challenging Unreal and Real Environments for Traffic Sign Recognition](https://mlsanity.com/paper/1712.02463/) (2017)
- [Are Adversarial Robustness and Common Perturbation Robustness Independent Attributes ?](https://mlsanity.com/paper/1909.02436/) (2019)
- [How is Gaze Influenced by Image Transformations? Dataset and Model](https://mlsanity.com/paper/1905.06803/) (2019)
- [How Image Degradations Affect Deep CNN-based Face Recognition?](https://mlsanity.com/paper/1608.05246/) (2016)

## Built on by (most cited first, 1 of 1)

- [Using Synthetic Corruptions to Measure Robustness to Natural Distribution Shifts](https://mlsanity.com/paper/2107.12052/) (2021)

## Abstract (arXiv)

Neural Networks are sensitive to various corruptions that usually occur in real-world applications such as blurs, noises, low-lighting conditions, etc. To estimate the robustness of neural networks to these common corruptions, we generally use a group of modeled corruptions gathered into a benchmark. Unfortunately, no objective criterion exists to determine whether a benchmark is representative of a large diversity of independent corruptions. In this paper, we propose a metric called corruption overlapping score, which can be used to reveal flaws in corruption benchmarks. Two corruptions overlap when the robustnesses of neural networks to these corruptions are correlated. We argue that taking into account overlappings between corruptions can help to improve existing benchmarks or build better ones.