# On Pitfalls of Measuring Occlusion Robustness through Data Distortion

arXiv: https://arxiv.org/abs/2211.13734 · HTML: https://mlsanity.com/paper/2211.13734/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 21 references that are in the map.
- First posted on arXiv 24 Nov 2022.

## Builds on (most cited first, 18 of 18)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation](https://mlsanity.com/paper/2008.03703/) (2020)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [Robust and Generalizable Visual Representation Learning via Random Convolutions](https://mlsanity.com/paper/2007.13003/) (2020)
- [Compositional Convolutional Neural Networks: A Robust and Interpretable Model for Object Recognition under Occlusion](https://mlsanity.com/paper/2006.15538/) (2020)
- [An Empirical Evaluation on Robustness and Uncertainty of Regularization Methods](https://mlsanity.com/paper/2003.03879/) (2020)
- [Robustness of Object Recognition under Extreme Occlusion in Humans and Computational Models](https://mlsanity.com/paper/1905.04598/) (2019)
- [FMix: Enhancing Mixed Sample Data Augmentation](https://mlsanity.com/paper/2002.12047/) (2020)

## Abstract (arXiv)

Over the past years, the crucial role of data has largely been shadowed by the field's focus on architectures and training procedures. We often cause changes to the data without being aware of their wider implications. In this paper we show that distorting images without accounting for the artefacts introduced leads to biased results when establishing occlusion robustness. To ensure models behave as expected in real-world scenarios, we need to rule out the impact added artefacts have on evaluation. We propose a new approach, iOcclusion, as a fairer alternative for applications where the possible occluders are unknown.