# Amodal Panoptic Segmentation

arXiv: https://arxiv.org/abs/2202.11542 · HTML: https://mlsanity.com/paper/2202.11542/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 42 references that are in the map.
- First posted on arXiv 23 Feb 2022.

## Builds on (most cited first, 30 of 33)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation](https://mlsanity.com/paper/2003.07853/) (2020)
- [MOTS: Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/1902.03604/) (2019)
- [UPSNet: A Unified Panoptic Segmentation Network](https://mlsanity.com/paper/1901.03784/) (2019)
- [Seamless Scene Segmentation](https://mlsanity.com/paper/1905.01220/) (2019)
- [EfficientPS: Efficient Panoptic Segmentation](https://mlsanity.com/paper/2004.02307/) (2020)
- [Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers](https://mlsanity.com/paper/2103.12340/) (2021)
- [Semantic Amodal Segmentation](https://mlsanity.com/paper/1509.01329/) (2015)
- [Searching for Efficient Multi-Scale Architectures for Dense Image Prediction](https://mlsanity.com/paper/1809.04184/) (2018)
- [SSAP: Single-Shot Instance Segmentation With Affinity Pyramid](https://mlsanity.com/paper/1909.01616/) (2019)
- [In-Place Activated BatchNorm for Memory-Optimized Training of DNNs](https://mlsanity.com/paper/1712.02616/) (2017)
- [EfficientLPS: Efficient LiDAR Panoptic Segmentation](https://mlsanity.com/paper/2102.08009/) (2021)
- [Amodal Instance Segmentation](https://mlsanity.com/paper/1604.08202/) (2016)
- [DeeperLab: Single-Shot Image Parser](https://mlsanity.com/paper/1902.05093/) (2019)
- [Learning to See the Invisible: End-to-End Trainable Amodal Instance Segmentation](https://mlsanity.com/paper/1804.08864/) (2018)
- [Bird's-Eye-View Panoptic Segmentation Using Monocular Frontal View Images](https://mlsanity.com/paper/2108.03227/) (2021)
- [There is More than Meets the Eye: Self-Supervised Multi-Object Detection and Tracking with Sound by Distilling Multimodal Knowledge](https://mlsanity.com/paper/2103.01353/) (2021)
- [Amodal Segmentation Based on Visible Region Segmentation and Shape Prior](https://mlsanity.com/paper/2012.05598/) (2020)
- [Self-Supervised Visual Terrain Classification from Unsupervised Acoustic Feature Learning](https://mlsanity.com/paper/1912.03227/) (2019)
- [Efficient Decomposition of Image and Mesh Graphs by Lifted Multicuts](https://mlsanity.com/paper/1505.06973/) (2015)
- [Loss Max-Pooling for Semantic Image Segmentation](https://mlsanity.com/paper/1704.02966/) (2017)

## Built on by (most cited first, 30 of 38)

- [Amodal Ground Truth and Completion in the Wild](https://mlsanity.com/paper/2312.17247/) (2023)
- [AISFormer: Amodal Instance Segmentation with Transformer](https://mlsanity.com/paper/2210.06323/) (2022)
- [Coarse-to-Fine Amodal Segmentation with Shape Prior](https://mlsanity.com/paper/2308.16825/) (2023)
- [Perceiving the Invisible: Proposal-Free Amodal Panoptic Segmentation](https://mlsanity.com/paper/2205.14637/) (2022)
- [Learning and Aggregating Lane Graphs for Urban Automated Driving](https://mlsanity.com/paper/2302.06175/) (2023)
- [Image Amodal Completion: A Survey](https://mlsanity.com/paper/2207.02062/) (2022)
- [CoDEPS: Online Continual Learning for Depth Estimation and Panoptic Segmentation](https://mlsanity.com/paper/2303.10147/) (2023)
- [Multi-label affordance mapping from egocentric vision](https://mlsanity.com/paper/2309.02120/) (2023)
- [Few-Shot Panoptic Segmentation With Foundation Models](https://mlsanity.com/paper/2309.10726/) (2023)
- [Uncertainty-aware Panoptic Segmentation](https://mlsanity.com/paper/2206.14554/) (2022)
- [Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation](https://mlsanity.com/paper/2307.06125/) (2023)
- [AmodalSynthDrive: A Synthetic Amodal Perception Dataset for Autonomous Driving](https://mlsanity.com/paper/2309.06547/) (2023)
- [Panoptic Out-of-Distribution Segmentation](https://mlsanity.com/paper/2310.11797/) (2023)
- [Syn-Mediverse: A Multimodal Synthetic Dataset for Intelligent Scene Understanding of Healthcare Facilities](https://mlsanity.com/paper/2308.03193/) (2023)
- [Self-Supervised Representation Learning from Temporal Ordering of Automated Driving Sequences](https://mlsanity.com/paper/2302.09043/) (2023)
- [EviPrompt: A Training-Free Evidential Prompt Generation Method for Segment Anything Model in Medical Images](https://mlsanity.com/paper/2311.06400/) (2023)
- [Amodal Optical Flow](https://mlsanity.com/paper/2311.07761/) (2023)
- [Occlusion-Aware Seamless Segmentation](https://mlsanity.com/paper/2407.02182/) (2024)
- [Segment Anything, Even Occluded](https://mlsanity.com/paper/2503.06261/) (2025)
- [On Hyperbolic Embeddings in 2D Object Detection](https://mlsanity.com/paper/2203.08049/) (2022)
- [INoD: Injected Noise Discriminator for Self-Supervised Representation Learning in Agricultural Fields](https://mlsanity.com/paper/2303.18101/) (2023)
- [ShapeFormer: Shape Prior Visible-to-Amodal Transformer-based Amodal Instance Segmentation](https://mlsanity.com/paper/2403.11376/) (2024)
- [GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherence](https://mlsanity.com/paper/2511.21945/) (2025)
- [Occ2Net: Robust Image Matching Based on 3D Occupancy Estimation for Occluded Regions](https://mlsanity.com/paper/2308.16160/) (2023)
- [Amodal Instance Segmentation with Diffusion Shape Prior Estimation](https://mlsanity.com/paper/2409.18256/) (2024)
- [Unlocking Constraints: Source-Free Occlusion-Aware Seamless Segmentation](https://mlsanity.com/paper/2506.21198/) (2025)
- [Asynchronous Events-based Panoptic Segmentation using Graph Mixer Neural Network](https://mlsanity.com/paper/2305.03640/) (2023)
- [Unveiling the Invisible: Reasoning Complex Occlusions Amodally with AURA](https://mlsanity.com/paper/2503.10225/) (2025)
- [Structure-Guided Image Completion with Image-level and Object-level Semantic Discriminators](https://mlsanity.com/paper/2212.06310/) (2022)
- [Amodal Intra-class Instance Segmentation: Synthetic Datasets and Benchmark](https://mlsanity.com/paper/2303.06596/) (2023)

## Abstract (arXiv)

Humans have the remarkable ability to perceive objects as a whole, even when parts of them are occluded. This ability of amodal perception forms the basis of our perceptual and cognitive understanding of our world. To enable robots to reason with this capability, we formulate and propose a novel task that we name amodal panoptic segmentation. The goal of this task is to simultaneously predict the pixel-wise semantic segmentation labels of the visible regions of stuff classes and the instance segmentation labels of both the visible and occluded regions of thing classes. To facilitate research on this new task, we extend two established benchmark datasets with pixel-level amodal panoptic segmentation labels that we make publicly available as KITTI-360-APS and BDD100K-APS. We present several strong baselines, along with the amodal panoptic quality (APQ) and amodal parsing coverage (APC) metrics to quantify the performance in an interpretable manner. Furthermore, we propose the novel amodal panoptic segmentation network (APSNet), as a first step towards addressing this task by explicitly modeling the complex relationships between the occluders and occludes. Extensive experimental evaluations demonstrate that APSNet achieves state-of-the-art performance on both benchmarks and more importantly exemplifies the utility of amodal recognition. The benchmarks are available at http://amodal-panoptic.cs.uni-freiburg.de.