# Combining Compositional Models and Deep Networks For Robust Object Classification under Occlusion

arXiv: https://arxiv.org/abs/1905.11826 · HTML: https://mlsanity.com/paper/1905.11826/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2019.

## Built on by (most cited first, 24 of 24)

- [Occluded Video Instance Segmentation: A Benchmark](https://mlsanity.com/paper/2102.01558/) (2021)
- [NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation](https://mlsanity.com/paper/2101.12378/) (2021)
- [3D-Aware Neural Body Fitting for Occlusion Robust 3D Human Pose Estimation](https://mlsanity.com/paper/2308.10123/) (2023)
- [Occlusion Handling in Generic Object Detection: A Review](https://mlsanity.com/paper/2101.08845/) (2021)
- [Robust Category-Level 6D Pose Estimation with Coarse-to-Fine Rendering of Neural Features](https://mlsanity.com/paper/2209.05624/) (2022)
- [Adversarial Patch Attacks and Defences in Vision-Based Tasks: A Survey](https://mlsanity.com/paper/2206.08304/) (2022)
- [Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose](https://mlsanity.com/paper/2110.14213/) (2021)
- [Explanations for Occluded Images](https://mlsanity.com/paper/2103.03622/) (2021)
- [OOD-CV-v2: An extended Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images](https://mlsanity.com/paper/2304.10266/) (2023)
- [NOVUM: Neural Object Volumes for Robust Object Classification](https://mlsanity.com/paper/2305.14668/) (2023)
- [Finding Differences Between Transformers and ConvNets Using Counterfactual Simulation Testing](https://mlsanity.com/paper/2211.16499/) (2022)
- [Mitigating the Effect of Incidental Correlations on Part-based Learning](https://mlsanity.com/paper/2310.00377/) (2023)
- [Are Deep Learning Models Robust to Partial Object Occlusion in Visual Recognition Tasks?](https://mlsanity.com/paper/2409.10775/) (2024)
- [CORL: Compositional Representation Learning for Few-Shot Classification](https://mlsanity.com/paper/2101.11878/) (2021)
- [FrameHopper: Selective Processing of Video Frames in Detection-driven Real-Time Video Analytics](https://mlsanity.com/paper/2203.11493/) (2022)
- [SeeThrough3D: Occlusion Aware 3D Control in Text-to-Image Generation](https://mlsanity.com/paper/2602.23359/) (2026)
- [Look Through Masks: Towards Masked Face Recognition with De-Occlusion Distillation](https://mlsanity.com/paper/2409.12385/) (2024)
- [D-Feat Occlusions: Diffusion Features for Robustness to Partial Visual Occlusions in Object Recognition](https://mlsanity.com/paper/2504.06432/) (2025)
- [BIFRNet: A Brain-Inspired Feature Restoration DNN for Partially Occluded Image Recognition](https://mlsanity.com/paper/2303.01309/) (2023)
- [Latent Enhancing AutoEncoder for Occluded Image Classification](https://mlsanity.com/paper/2402.06936/) (2024)
- [A Parts Based Registration Loss for Detecting Knee Joint Areas](https://mlsanity.com/paper/2307.00083/) (2023)
- [Compositional Representation Learning for Brain Tumour Segmentation](https://mlsanity.com/paper/2310.06562/) (2023)
- [HB-net: Holistic bursting cell cluster integrated network for occluded multi-objects recognition](https://mlsanity.com/paper/2310.11834/) (2023)
- [DINeMo: Learning Neural Mesh Models with no 3D Annotations](https://mlsanity.com/paper/2503.20220/) (2025)

## Abstract (arXiv)

Deep convolutional neural networks (DCNNs) are powerful models that yield impressive results at object classification. However, recent work has shown that they do not generalize well to partially occluded objects and to mask attacks. In contrast to DCNNs, compositional models are robust to partial occlusion, however, they are not as discriminative as deep models. In this work, we combine DCNNs and compositional object models to retain the best of both approaches: a discriminative model that is robust to partial occlusion and mask attacks. Our model is learned in two steps. First, a standard DCNN is trained for image classification. Subsequently, we cluster the DCNN features into dictionaries. We show that the dictionary components resemble object part detectors and learn the spatial distribution of parts for each object class. We propose mixtures of compositional models to account for large changes in the spatial activation patterns (e.g. due to changes in the 3D pose of an object). At runtime, an image is first classified by the DCNN in a feedforward manner. The prediction uncertainty is used to detect partially occluded objects, which in turn are classified by the compositional model. Our experimental results demonstrate that combining compositional models and DCNNs resolves a fundamental problem of current deep learning approaches to computer vision: The combined model recognizes occluded objects, even when it has not been exposed to occluded objects during training, while at the same time maintaining high discriminative performance for non-occluded objects.