# Robust Instance Segmentation through Reasoning about Multi-Object Occlusion

arXiv: https://arxiv.org/abs/2012.02107 · HTML: https://mlsanity.com/paper/2012.02107/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2020.

## Built on by (most cited first, 23 of 23)

- [DeepSportradar-v1: Computer Vision Dataset for Sports Understanding with High Quality Annotations](https://mlsanity.com/paper/2208.08190/) (2022)
- [Instance-wise Occlusion and Depth Orders in Natural Scenes](https://mlsanity.com/paper/2111.14562/) (2021)
- [UMC: A Unified Bandwidth-efficient and Multi-resolution based Collaborative Perception Framework](https://mlsanity.com/paper/2303.12400/) (2023)
- [Robust Category-Level 6D Pose Estimation with Coarse-to-Fine Rendering of Neural Features](https://mlsanity.com/paper/2209.05624/) (2022)
- [Learning Environment-Aware Affordance for 3D Articulated Object Manipulation under Occlusions](https://mlsanity.com/paper/2309.07510/) (2023)
- [A Tri-Layer Plugin to Improve Occluded Detection](https://mlsanity.com/paper/2210.10046/) (2022)
- [Image Amodal Completion: A Survey](https://mlsanity.com/paper/2207.02062/) (2022)
- [Object-level Scene Deocclusion](https://mlsanity.com/paper/2406.07706/) (2024)
- [3D-Aware Visual Question Answering about Parts, Poses and Occlusions](https://mlsanity.com/paper/2310.17914/) (2023)
- [Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose](https://mlsanity.com/paper/2110.14213/) (2021)
- [vMFNet: Compositionality Meets Domain-generalised Segmentation](https://mlsanity.com/paper/2206.14538/) (2022)
- [Track Initialization and Re-Identification for~3D Multi-View Multi-Object Tracking](https://mlsanity.com/paper/2405.18606/) (2024)
- [Occlusion-Aware Seamless Segmentation](https://mlsanity.com/paper/2407.02182/) (2024)
- [A Bayesian Approach to OOD Robustness in Image Classification](https://mlsanity.com/paper/2403.07277/) (2024)
- [WALT3D: Generating Realistic Training Data from Time-Lapse Imagery for Reconstructing Dynamic Objects under Occlusion](https://mlsanity.com/paper/2403.19022/) (2024)
- [COCO-OLAC: A Benchmark for Occluded Panoptic Segmentation and Image Understanding](https://mlsanity.com/paper/2409.12760/) (2024)
- [Compositionally Equivariant Representation Learning](https://mlsanity.com/paper/2306.07783/) (2023)
- [Sequential Amodal Segmentation via Cumulative Occlusion Learning](https://mlsanity.com/paper/2405.05791/) (2024)
- [Detection Selection Algorithm: A Likelihood based Optimization Method to Perform Post Processing for Object Detection](https://mlsanity.com/paper/2212.05706/) (2022)
- [Compositional Representation Learning for Brain Tumour Segmentation](https://mlsanity.com/paper/2310.06562/) (2023)
- [Hybrid Deep Learning-Based for Enhanced Occlusion Segmentation in PICU Patient Monitoring](https://mlsanity.com/paper/2407.13341/) (2024)
- [Occlusion-Ordered Semantic Instance Segmentation](https://mlsanity.com/paper/2504.14054/) (2025)
- [A Survey on Deep Multi-Task Learning in Connected Autonomous Vehicles](https://mlsanity.com/paper/2508.00917/) (2025)

## Abstract (arXiv)

Analyzing complex scenes with Deep Neural Networks is a challenging task, particularly when images contain multiple objects that partially occlude each other. Existing approaches to image analysis mostly process objects independently and do not take into account the relative occlusion of nearby objects. In this paper, we propose a deep network for multi-object instance segmentation that is robust to occlusion and can be trained from bounding box supervision only. Our work builds on Compositional Networks, which learn a generative model of neural feature activations to locate occluders and to classify objects based on their non-occluded parts. We extend their generative model to include multiple objects and introduce a framework for efficient inference in challenging occlusion scenarios. In particular, we obtain feed-forward predictions of the object classes and their instance and occluder segmentations. We introduce an Occlusion Reasoning Module (ORM) that locates erroneous segmentations and estimates the occlusion order to correct them. The improved segmentation masks are, in turn, integrated into the network in a top-down manner to improve the image classification. Our experiments on the KITTI INStance dataset (KINS) and a synthetic occlusion dataset demonstrate the effectiveness and robustness of our model at multi-object instance segmentation under occlusion. Code is publically available at https://github.com/XD7479/Multi-Object-Occlusion.