# Compositional Convolutional Neural Networks: A Deep Architecture with Innate Robustness to Partial Occlusion

arXiv: https://arxiv.org/abs/2003.04490 · HTML: https://mlsanity.com/paper/2003.04490/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2020.

## Built on by (most cited first, 30 of 47)

- [Occluded Video Instance Segmentation: A Benchmark](https://mlsanity.com/paper/2102.01558/) (2021)
- [PatchCleanser: Certifiably Robust Defense against Adversarial Patches for Any Image Classifier](https://mlsanity.com/paper/2108.09135/) (2021)
- [Using latent space regression to analyze and leverage compositionality in GANs](https://mlsanity.com/paper/2103.10426/) (2021)
- [NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation](https://mlsanity.com/paper/2101.12378/) (2021)
- [SwapMix: Diagnosing and Regularizing the Over-Reliance on Visual Context in Visual Question Answering](https://mlsanity.com/paper/2204.02285/) (2022)
- [Occlusion-Aware Video Object Inpainting](https://mlsanity.com/paper/2108.06765/) (2021)
- [3D-Aware Neural Body Fitting for Occlusion Robust 3D Human Pose Estimation](https://mlsanity.com/paper/2308.10123/) (2023)
- [Occlusion Handling in Generic Object Detection: A Review](https://mlsanity.com/paper/2101.08845/) (2021)
- [Robust Category-Level 6D Pose Estimation with Coarse-to-Fine Rendering of Neural Features](https://mlsanity.com/paper/2209.05624/) (2022)
- [Image Amodal Completion: A Survey](https://mlsanity.com/paper/2207.02062/) (2022)
- [Adversarial Patch Attacks and Defences in Vision-Based Tasks: A Survey](https://mlsanity.com/paper/2206.08304/) (2022)
- [Interpretable Compositional Convolutional Neural Networks](https://mlsanity.com/paper/2107.04474/) (2021)
- [Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose](https://mlsanity.com/paper/2110.14213/) (2021)
- [Explanations for Occluded Images](https://mlsanity.com/paper/2103.03622/) (2021)
- [OOD-CV-v2: An extended Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images](https://mlsanity.com/paper/2304.10266/) (2023)
- [vMFNet: Compositionality Meets Domain-generalised Segmentation](https://mlsanity.com/paper/2206.14538/) (2022)
- [Soft Augmentation for Image Classification](https://mlsanity.com/paper/2211.04625/) (2022)
- [NOVUM: Neural Object Volumes for Robust Object Classification](https://mlsanity.com/paper/2305.14668/) (2023)
- [Compositor: Bottom-up Clustering and Compositing for Robust Part and Object Segmentation](https://mlsanity.com/paper/2306.07404/) (2023)
- [BLADE: Box-Level Supervised Amodal Segmentation through Directed Expansion](https://mlsanity.com/paper/2401.01642/) (2024)
- [OGMN: Occlusion-guided Multi-task Network for Object Detection in UAV Images](https://mlsanity.com/paper/2304.11805/) (2023)
- [O2RNet: Occluder-Occludee Relational Network for Robust Apple Detection in Clustered Orchard Environments](https://mlsanity.com/paper/2303.04884/) (2023)
- [Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimation](https://mlsanity.com/paper/2401.10848/) (2024)
- [Finding Differences Between Transformers and ConvNets Using Counterfactual Simulation Testing](https://mlsanity.com/paper/2211.16499/) (2022)
- [On Occlusions in Video Action Detection: Benchmark Datasets And Training Recipes](https://mlsanity.com/paper/2410.19553/) (2024)
- [Unsupervised Learning of Category-Level 3D Pose from Object-Centric Videos](https://mlsanity.com/paper/2407.04384/) (2024)
- [Are Deep Learning Models Robust to Partial Object Occlusion in Visual Recognition Tasks?](https://mlsanity.com/paper/2409.10775/) (2024)
- [CORL: Compositional Representation Learning for Few-Shot Classification](https://mlsanity.com/paper/2101.11878/) (2021)
- [PLUG: Revisiting Amodal Segmentation with Foundation Model and Hierarchical Focus](https://mlsanity.com/paper/2405.16094/) (2024)
- [A Bayesian Approach to OOD Robustness in Image Classification](https://mlsanity.com/paper/2403.07277/) (2024)

## Abstract (arXiv)

Recent findings show that deep convolutional neural networks (DCNNs) do not generalize well under partial occlusion. Inspired by the success of compositional models at classifying partially occluded objects, we propose to integrate compositional models and DCNNs into a unified deep model with innate robustness to partial occlusion. We term this architecture Compositional Convolutional Neural Network. In particular, we propose to replace the fully connected classification head of a DCNN with a differentiable compositional model. The generative nature of the compositional model enables it to localize occluders and subsequently focus on the non-occluded parts of the object. We conduct classification experiments on artificially occluded images as well as real images of partially occluded objects from the MS-COCO dataset. The results show that DCNNs do not classify occluded objects robustly, even when trained with data that is strongly augmented with partial occlusions. Our proposed model outperforms standard DCNNs by a large margin at classifying partially occluded objects, even when it has not been exposed to occluded objects during training. Additional experiments demonstrate that CompositionalNets can also localize the occluders accurately, despite being trained with class labels only. The code used in this work is publicly available.