# Weakly- and Semi-Supervised Panoptic Segmentation

arXiv: https://arxiv.org/abs/1808.03575 · HTML: https://mlsanity.com/paper/1808.03575/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Aug 2018.

## Built on by (most cited first, 30 of 42)

- [Mixed supervision for surface-defect detection: from weakly to fully supervised learning](https://mlsanity.com/paper/2104.06064/) (2021)
- [Adaptive Early-Learning Correction for Segmentation from Noisy Annotations](https://mlsanity.com/paper/2110.03740/) (2021)
- [Bird's-Eye-View Panoptic Segmentation Using Monocular Frontal View Images](https://mlsanity.com/paper/2108.03227/) (2021)
- [Affinity Attention Graph Neural Network for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2106.04054/) (2021)
- [A Survey on Label-efficient Deep Image Segmentation: Bridging the Gap between Weak Supervision and Dense Prediction](https://mlsanity.com/paper/2207.01223/) (2022)
- [Exemplar-Based Open-Set Panoptic Segmentation Network](https://mlsanity.com/paper/2105.08336/) (2021)
- [Fully Convolutional Networks for Panoptic Segmentation with Point-based Supervision](https://mlsanity.com/paper/2108.07682/) (2021)
- [DaTaSeg: Taming a Universal Multi-Dataset Multi-Task Segmentation Model](https://mlsanity.com/paper/2306.01736/) (2023)
- [Slot-VPS: Object-centric Representation Learning for Video Panoptic Segmentation](https://mlsanity.com/paper/2112.08949/) (2021)
- [Point2Mask: Point-supervised Panoptic Segmentation via Optimal Transport](https://mlsanity.com/paper/2308.01779/) (2023)
- [Pointly-Supervised Panoptic Segmentation](https://mlsanity.com/paper/2210.13950/) (2022)
- [Instance and Panoptic Segmentation Using Conditional Convolutions](https://mlsanity.com/paper/2102.03026/) (2021)
- [Panoptic Segmentation: A Review](https://mlsanity.com/paper/2111.10250/) (2021)
- [Hierarchical Deep Network with Uncertainty-aware Semi-supervised Learning for Vessel Segmentation](https://mlsanity.com/paper/2105.14732/) (2021)
- [Sparse Object-level Supervision for Instance Segmentation with Pixel Embeddings](https://mlsanity.com/paper/2103.14572/) (2021)
- [Combinatorial Optimization for Panoptic Segmentation: A Fully Differentiable Approach](https://mlsanity.com/paper/2106.03188/) (2021)
- [Inferring the Class Conditional Response Map for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2110.14309/) (2021)
- [Adversarial Dual-Student with Differentiable Spatial Warping for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.02792/) (2022)
- [Panoptic, Instance and Semantic Relations: A Relational Context Encoder to Enhance Panoptic Segmentation](https://mlsanity.com/paper/2204.05370/) (2022)
- [Multi-task Fusion for Efficient Panoptic-Part Segmentation](https://mlsanity.com/paper/2212.07671/) (2022)
- [Cyclic Learning: Bridging Image-level Labels and Nuclei Instance Segmentation](https://mlsanity.com/paper/2306.02691/) (2023)
- [Hierarchical Lov'asz Embeddings for Proposal-free Panoptic Segmentation](https://mlsanity.com/paper/2106.04555/) (2021)
- [TRACE: Your Diffusion Model is Secretly an Instance Edge Detector](https://mlsanity.com/paper/2503.07982/) (2025)
- [Point-DETR3D: Leveraging Imagery Data with Spatial Point Prior for Weakly Semi-supervised 3D Object Detection](https://mlsanity.com/paper/2403.15317/) (2024)
- [DEARLi: Decoupled Enhancement of Recognition and Localization for Semi-supervised Panoptic Segmentation](https://mlsanity.com/paper/2507.10118/) (2025)
- [Weakly Supervised Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/2101.00667/) (2021)
- [Training Semantic Segmentation on Heterogeneous Datasets](https://mlsanity.com/paper/2301.07634/) (2023)
- [JPPF: Multi-task Fusion for Consistent Panoptic-Part Segmentation](https://mlsanity.com/paper/2311.18618/) (2023)
- [MetaSeg: Content-Aware Meta-Net for Omni-Supervised Semantic Segmentation](https://mlsanity.com/paper/2401.11738/) (2024)
- [Visual Boundary Knowledge Translation for Foreground Segmentation](https://mlsanity.com/paper/2108.00379/) (2021)

## Abstract (arXiv)

We present a weakly supervised model that jointly performs both semantic- and instance-segmentation -- a particularly relevant problem given the substantial cost of obtaining pixel-perfect annotation for these tasks. In contrast to many popular instance segmentation approaches based on object detectors, our method does not predict any overlapping instances. Moreover, we are able to segment both "thing" and "stuff" classes, and thus explain all the pixels in the image. "Thing" classes are weakly-supervised with bounding boxes, and "stuff" with image-level tags. We obtain state-of-the-art results on Pascal VOC, for both full and weak supervision (which achieves about 95% of fully-supervised performance). Furthermore, we present the first weakly-supervised results on Cityscapes for both semantic- and instance-segmentation. Finally, we use our weakly supervised framework to analyse the relationship between annotation quality and predictive performance, which is of interest to dataset creators.