# Self-Supervised Scene De-occlusion

arXiv: https://arxiv.org/abs/2004.02788 · HTML: https://mlsanity.com/paper/2004.02788/

## Key facts

- Cited by 96 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2020.

## Built on by (most cited first, 30 of 96)

- [MOSE: A New Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2302.01872/) (2023)
- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [Occluded Video Instance Segmentation: A Benchmark](https://mlsanity.com/paper/2102.01558/) (2021)
- [Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesis](https://mlsanity.com/paper/2405.14868/) (2024)
- [Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers](https://mlsanity.com/paper/2103.12340/) (2021)
- [pix2gestalt: Amodal Segmentation by Synthesizing Wholes](https://mlsanity.com/paper/2401.14398/) (2024)
- [Making Images Real Again: A Comprehensive Survey on Deep Image Composition](https://mlsanity.com/paper/2106.14490/) (2021)
- [Segmenting Moving Objects via an Object-Centric Layered Representation](https://mlsanity.com/paper/2207.02206/) (2022)
- [DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videos](https://mlsanity.com/paper/2405.02280/) (2024)
- [MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2508.05630/) (2025)
- [HoloPart: Generative 3D Part Amodal Segmentation](https://mlsanity.com/paper/2504.07943/) (2025)
- [Amodal3R: Amodal 3D Reconstruction from Occluded 2D Images](https://mlsanity.com/paper/2503.13439/) (2025)
- [Amodal Ground Truth and Completion in the Wild](https://mlsanity.com/paper/2312.17247/) (2023)
- [MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image Generation](https://mlsanity.com/paper/2404.02790/) (2024)
- [Reduce Information Loss in Transformers for Pluralistic Image Inpainting](https://mlsanity.com/paper/2205.05076/) (2022)
- [3D Interacting Hand Pose Estimation by Hand De-occlusion and Removal](https://mlsanity.com/paper/2207.11061/) (2022)
- [AISFormer: Amodal Instance Segmentation with Transformer](https://mlsanity.com/paper/2210.06323/) (2022)
- [Qwen-Image-Layered: Towards Inherent Editability via Layer Decomposition](https://mlsanity.com/paper/2512.15603/) (2025)
- [Human De-occlusion: Invisible Perception and Recovery for Humans](https://mlsanity.com/paper/2103.11597/) (2021)
- [Instance-wise Occlusion and Depth Orders in Natural Scenes](https://mlsanity.com/paper/2111.14562/) (2021)
- [Occlusion-Aware Video Object Inpainting](https://mlsanity.com/paper/2108.06765/) (2021)
- [Coarse-to-Fine Amodal Segmentation with Shape Prior](https://mlsanity.com/paper/2308.16825/) (2023)
- [Occlusion Handling in Generic Object Detection: A Review](https://mlsanity.com/paper/2101.08845/) (2021)
- [Visiting the Invisible: Layer-by-Layer Completed Scene Decomposition](https://mlsanity.com/paper/2104.05367/) (2021)
- [Learning Environment-Aware Affordance for 3D Articulated Object Manipulation under Occlusions](https://mlsanity.com/paper/2309.07510/) (2023)
- [A Weakly Supervised Amodal Segmenter with Boundary Uncertainty Estimation](https://mlsanity.com/paper/2108.09897/) (2021)
- [A Tri-Layer Plugin to Improve Occluded Detection](https://mlsanity.com/paper/2210.10046/) (2022)
- [LayerD: Decomposing Raster Graphic Designs into Layers](https://mlsanity.com/paper/2509.25134/) (2025)
- [Open-World Amodal Appearance Completion](https://mlsanity.com/paper/2411.13019/) (2024)
- [Image Amodal Completion: A Survey](https://mlsanity.com/paper/2207.02062/) (2022)

## Abstract (arXiv)

Natural scene understanding is a challenging task, particularly when encountering images of multiple objects that are partially occluded. This obstacle is given rise by varying object ordering and positioning. Existing scene understanding paradigms are able to parse only the visible parts, resulting in incomplete and unstructured scene interpretation. In this paper, we investigate the problem of scene de-occlusion, which aims to recover the underlying occlusion ordering and complete the invisible parts of occluded objects. We make the first attempt to address the problem through a novel and unified framework that recovers hidden scene structures without ordering and amodal annotations as supervisions. This is achieved via Partial Completion Network (PCNet)-mask (M) and -content (C), that learn to recover fractions of object masks and contents, respectively, in a self-supervised manner. Based on PCNet-M and PCNet-C, we devise a novel inference scheme to accomplish scene de-occlusion, via progressive ordering recovery, amodal completion and content completion. Extensive experiments on real-world scenes demonstrate the superior performance of our approach to other alternatives. Remarkably, our approach that is trained in a self-supervised manner achieves comparable results to fully-supervised methods. The proposed scene de-occlusion framework benefits many applications, including high-quality and controllable image manipulation and scene recomposition (see Fig. 1), as well as the conversion of existing modal mask annotations to amodal mask annotations.