# Robustness of Object Recognition under Extreme Occlusion in Humans and Computational Models

arXiv: https://arxiv.org/abs/1905.04598 · HTML: https://mlsanity.com/paper/1905.04598/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2019.

## Built on by (most cited first, 18 of 18)

- [Occlusion Handling in Generic Object Detection: A Review](https://mlsanity.com/paper/2101.08845/) (2021)
- [Image Amodal Completion: A Survey](https://mlsanity.com/paper/2207.02062/) (2022)
- [Adversarial Patch Attacks and Defences in Vision-Based Tasks: A Survey](https://mlsanity.com/paper/2206.08304/) (2022)
- [Explanations for Occluded Images](https://mlsanity.com/paper/2103.03622/) (2021)
- [Soft Augmentation for Image Classification](https://mlsanity.com/paper/2211.04625/) (2022)
- [Finding Differences Between Transformers and ConvNets Using Counterfactual Simulation Testing](https://mlsanity.com/paper/2211.16499/) (2022)
- [OccRob: Efficient SMT-Based Occlusion Robustness Verification of Deep Neural Networks](https://mlsanity.com/paper/2301.11912/) (2023)
- [Are Deep Learning Models Robust to Partial Object Occlusion in Visual Recognition Tasks?](https://mlsanity.com/paper/2409.10775/) (2024)
- [NOMAD: A Natural, Occluded, Multi-scale Aerial Dataset, for Emergency Response Scenarios](https://mlsanity.com/paper/2309.09518/) (2023)
- [Analyzing Effects of Mixed Sample Data Augmentation on Model Interpretability](https://mlsanity.com/paper/2303.14608/) (2023)
- [Now You See Me: Robust approach to Partial Occlusions](https://mlsanity.com/paper/2304.11779/) (2023)
- [D-Feat Occlusions: Diffusion Features for Robustness to Partial Visual Occlusions in Object Recognition](https://mlsanity.com/paper/2504.06432/) (2025)
- [Hardwiring ViT Patch Selectivity into CNNs using Patch Mixing](https://mlsanity.com/paper/2306.17848/) (2023)
- [On Pitfalls of Measuring Occlusion Robustness through Data Distortion](https://mlsanity.com/paper/2211.13734/) (2022)
- [Track Anything Behind Everything: Zero-Shot Amodal Video Object Segmentation](https://mlsanity.com/paper/2411.19210/) (2024)
- [Psych-Occlusion: Using Visual Psychophysics for Aerial Detection of Occluded Persons during Search and Rescue](https://mlsanity.com/paper/2412.05553/) (2024)
- [Combined Image Data Augmentations diminish the benefits of Adaptive Label Smoothing](https://mlsanity.com/paper/2507.16427/) (2025)
- [Tiny Collaborative Inference for Occlusion-Robust Object Detection](https://mlsanity.com/paper/2606.02894/) (2026)

## Abstract (arXiv)

Most objects in the visual world are partially occluded, but humans can recognize them without difficulty. However, it remains unknown whether object recognition models like convolutional neural networks (CNNs) can handle real-world occlusion. It is also a question whether efforts to make these models robust to constant mask occlusion are effective for real-world occlusion. We test both humans and the above-mentioned computational models in a challenging task of object recognition under extreme occlusion, where target objects are heavily occluded by irrelevant real objects in real backgrounds. Our results show that human vision is very robust to extreme occlusion while CNNs are not, even with modifications to handle constant mask occlusion. This implies that the ability to handle constant mask occlusion does not entail robustness to real-world occlusion. As a comparison, we propose another computational model that utilizes object parts/subparts in a compositional manner to build robustness to occlusion. This performs significantly better than CNN-based models on our task with error patterns similar to humans. These findings suggest that testing under extreme occlusion can better reveal the robustness of visual recognition, and that the principle of composition can encourage such robustness.