# A Tri-Layer Plugin to Improve Occluded Detection

arXiv: https://arxiv.org/abs/2210.10046 · HTML: https://mlsanity.com/paper/2210.10046/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 40 references that are in the map.
- First posted on arXiv 18 Oct 2022.

## Builds on (most cited first, 26 of 26)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Occluded Video Instance Segmentation: A Benchmark](https://mlsanity.com/paper/2102.01558/) (2021)
- [Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers](https://mlsanity.com/paper/2103.12340/) (2021)
- [Semantic Amodal Segmentation](https://mlsanity.com/paper/1509.01329/) (2015)
- [UA-DETRAC: A New Benchmark and Protocol for Multi-Object Detection and Tracking](https://mlsanity.com/paper/1511.04136/) (2015)
- [Self-Supervised Scene De-occlusion](https://mlsanity.com/paper/2004.02788/) (2020)
- [Amodal Instance Segmentation](https://mlsanity.com/paper/1604.08202/) (2016)
- [Learning to See the Invisible: End-to-End Trainable Amodal Instance Segmentation](https://mlsanity.com/paper/1804.08864/) (2018)
- [Robust Object Detection under Occlusion with Context-Aware CompositionalNets](https://mlsanity.com/paper/2005.11643/) (2020)
- [Amodal Segmentation through Out-of-Task and Out-of-Distribution Generalization with a Bayesian Model](https://mlsanity.com/paper/2010.13175/) (2020)
- [Occlusion Handling in Generic Object Detection: A Review](https://mlsanity.com/paper/2101.08845/) (2021)
- [Robust Instance Segmentation through Reasoning about Multi-Object Occlusion](https://mlsanity.com/paper/2012.02107/) (2020)
- [MessyTable: Instance Association in Multiple Camera Views](https://mlsanity.com/paper/2007.14878/) (2020)
- [Transferable Active Grasping and Real Embodied Dataset](https://mlsanity.com/paper/2004.13358/) (2020)

## Built on by (most cited first, 21 of 21)

- [InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environment](https://mlsanity.com/paper/2406.04882/) (2024)
- [Bird's-Eye-View Scene Graph for Vision-Language Navigation](https://mlsanity.com/paper/2308.04758/) (2023)
- [pix2gestalt: Amodal Segmentation by Synthesizing Wholes](https://mlsanity.com/paper/2401.14398/) (2024)
- [MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2508.05630/) (2025)
- [Amodal Ground Truth and Completion in the Wild](https://mlsanity.com/paper/2312.17247/) (2023)
- [When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models](https://mlsanity.com/paper/2405.10255/) (2024)
- [A General Protocol to Probe Large Vision Models for 3D Physical Understanding](https://mlsanity.com/paper/2310.06836/) (2023)
- [Learning Environment-Aware Affordance for 3D Articulated Object Manipulation under Occlusions](https://mlsanity.com/paper/2309.07510/) (2023)
- [Efficient Image Generation with Variadic Attention Heads](https://mlsanity.com/paper/2211.05770/) (2022)
- [Tracking through Containers and Occluders in the Wild](https://mlsanity.com/paper/2305.03052/) (2023)
- [LaRender: Training-Free Occlusion Control in Image Generation via Latent Rendering](https://mlsanity.com/paper/2508.07647/) (2025)
- [ELIP: Enhanced Visual-Language Foundation Models for Image Retrieval](https://mlsanity.com/paper/2502.15682/) (2025)
- [On Occlusions in Video Action Detection: Benchmark Datasets And Training Recipes](https://mlsanity.com/paper/2410.19553/) (2024)
- [Inferring Dynamic Physical Properties from Video Foundation Models](https://mlsanity.com/paper/2510.02311/) (2025)
- [Extreme Point Supervised Instance Segmentation](https://mlsanity.com/paper/2405.20729/) (2024)
- [TARGO: Benchmarking Target-driven Object Grasping under Occlusions](https://mlsanity.com/paper/2407.06168/) (2024)
- [COCO-OLAC: A Benchmark for Occluded Panoptic Segmentation and Image Understanding](https://mlsanity.com/paper/2409.12760/) (2024)
- ["It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Models](https://mlsanity.com/paper/2511.08917/) (2025)
- [Latent Enhancing AutoEncoder for Occluded Image Classification](https://mlsanity.com/paper/2402.06936/) (2024)
- [Improving Classification of Occluded Objects through Scene Context](https://mlsanity.com/paper/2510.26681/) (2025)
- [Integrating Multimodal Large Language Model Knowledge into Amodal Completion](https://mlsanity.com/paper/2603.28333/) (2026)

## Abstract (arXiv)

Detecting occluded objects still remains a challenge for state-of-the-art object detectors. The objective of this work is to improve the detection for such objects, and thereby improve the overall performance of a modern object detector. To this end we make the following four contributions: (1) We propose a simple 'plugin' module for the detection head of two-stage object detectors to improve the recall of partially occluded objects. The module predicts a tri-layer of segmentation masks for the target object, the occluder and the occludee, and by doing so is able to better predict the mask of the target object. (2) We propose a scalable pipeline for generating training data for the module by using amodal completion of existing object detection and instance segmentation training datasets to establish occlusion relationships. (3) We also establish a COCO evaluation dataset to measure the recall performance of partially occluded and separated objects. (4) We show that the plugin module inserted into a two-stage detector can boost the performance significantly, by only fine-tuning the detection head, and with additional improvements if the entire architecture is fine-tuned. COCO results are reported for Mask R-CNN with Swin-T or Swin-S backbones, and Cascade Mask R-CNN with a Swin-B backbone.