# SeGAN: Segmenting and Generating the Invisible

arXiv: https://arxiv.org/abs/1703.10239 · HTML: https://mlsanity.com/paper/1703.10239/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2017.

## Built on by (most cited first, 30 of 60)

- [ProcTHOR: Large-Scale Embodied AI Using Procedural Generation](https://mlsanity.com/paper/2206.06994/) (2022)
- [Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesis](https://mlsanity.com/paper/2405.14868/) (2024)
- [Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers](https://mlsanity.com/paper/2103.12340/) (2021)
- [pix2gestalt: Amodal Segmentation by Synthesizing Wholes](https://mlsanity.com/paper/2401.14398/) (2024)
- [Making Images Real Again: A Comprehensive Survey on Deep Image Composition](https://mlsanity.com/paper/2106.14490/) (2021)
- [DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videos](https://mlsanity.com/paper/2405.02280/) (2024)
- [HoloPart: Generative 3D Part Amodal Segmentation](https://mlsanity.com/paper/2504.07943/) (2025)
- [Amodal Ground Truth and Completion in the Wild](https://mlsanity.com/paper/2312.17247/) (2023)
- [MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image Generation](https://mlsanity.com/paper/2404.02790/) (2024)
- [3D Interacting Hand Pose Estimation by Hand De-occlusion and Removal](https://mlsanity.com/paper/2207.11061/) (2022)
- [Human De-occlusion: Invisible Perception and Recovery for Humans](https://mlsanity.com/paper/2103.11597/) (2021)
- [Occlusion-Aware Video Object Inpainting](https://mlsanity.com/paper/2108.06765/) (2021)
- [Occlusion Handling in Generic Object Detection: A Review](https://mlsanity.com/paper/2101.08845/) (2021)
- [Visiting the Invisible: Layer-by-Layer Completed Scene Decomposition](https://mlsanity.com/paper/2104.05367/) (2021)
- [A Weakly Supervised Amodal Segmenter with Boundary Uncertainty Estimation](https://mlsanity.com/paper/2108.09897/) (2021)
- [Open-World Amodal Appearance Completion](https://mlsanity.com/paper/2411.13019/) (2024)
- [Image Amodal Completion: A Survey](https://mlsanity.com/paper/2207.02062/) (2022)
- [Object-level Scene Deocclusion](https://mlsanity.com/paper/2406.07706/) (2024)
- [Using Diffusion Priors for Video Amodal Segmentation](https://mlsanity.com/paper/2412.04623/) (2024)
- [Spatiotemporal Data Mining: A Survey on Challenges and Open Problems](https://mlsanity.com/paper/2103.17128/) (2021)
- [Exploring Phrase-Level Grounding with Text-to-Image Diffusion Model](https://mlsanity.com/paper/2407.05352/) (2024)
- [Generative Adversarial Networks for Image Super-Resolution: A Survey](https://mlsanity.com/paper/2204.13620/) (2022)
- [BLADE: Box-Level Supervised Amodal Segmentation through Directed Expansion](https://mlsanity.com/paper/2401.01642/) (2024)
- [Beyond the Visible: Benchmarking Occlusion Perception in Multimodal Large Language Models](https://mlsanity.com/paper/2508.04059/) (2025)
- [Behind the leaves -- Estimation of occluded grapevine berries with conditional generative adversarial networks](https://mlsanity.com/paper/2105.10325/) (2021)
- [Amodal Optical Flow](https://mlsanity.com/paper/2311.07761/) (2023)
- [Neural Radiance Field Codebooks](https://mlsanity.com/paper/2301.04101/) (2023)
- [TAO-Amodal: A Benchmark for Tracking Any Object Amodally](https://mlsanity.com/paper/2312.12433/) (2023)
- [Segment Anything, Even Occluded](https://mlsanity.com/paper/2503.06261/) (2025)
- [Occlusion-Aware Instance Segmentation via BiLayer Network Architectures](https://mlsanity.com/paper/2208.04438/) (2022)

## Abstract (arXiv)

Objects often occlude each other in scenes; Inferring their appearance beyond their visible parts plays an important role in scene understanding, depth estimation, object interaction and manipulation. In this paper, we study the challenging problem of completing the appearance of occluded objects. Doing so requires knowing which pixels to paint (segmenting the invisible parts of objects) and what color to paint them (generating the invisible parts). Our proposed novel solution, SeGAN, jointly optimizes for both segmentation and generation of the invisible parts of objects. Our experimental results show that: (a) SeGAN can learn to generate the appearance of the occluded parts of objects; (b) SeGAN outperforms state-of-the-art segmentation baselines for the invisible parts of objects; (c) trained on synthetic photo realistic images, SeGAN can reliably segment natural images; (d) by reasoning about occluder occludee relations, our method can infer depth layering.