Attention-guided Unified Network for Panoptic Segmentation
Not on the map. Why?
Abstract (arXiv)
This paper studies panoptic segmentation, a recently proposed task which segments foreground (FG) objects at the instance level as well as background (BG) contents at the semantic level. Existing methods mostly dealt with these two problems separately, but in this paper, we reveal the underlying relationship between them, in particular, FG objects provide complementary cues to assist BG understanding. Our approach, named the Attention-guided Unified Network (AUNet), is a unified framework with two branches for FG and BG segmentation simultaneously. Two sources of attentions are added to the BG branch, namely, RPN and FG segmentation mask to provide object-level and pixel-level attentions, respectively. Our approach is generalized to different backbones with consistent accuracy gain in both FG and BG segmentation, and also sets new state-of-the-arts both in the MS-COCO (46.5% PQ) and Cityscapes (59.0% PQ) benchmarks.
Key facts. Cited by 71 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Dec 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 19; 2022: 23; 2023: 17; 2024: 7; 2025: 4; 2026: 1.
71 papers.
Sort:
By area:
- 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
- 2023Personalize Segment Anything Model with One Shotcited by 250
- 2022Open-Vocabulary Universal Image Segmentation with MaskCLIPcited by 132
- 2021Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformerscited by 109
- 2021Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentationcited by 107
- 2022A Generalist Framework for Panoptic Segmentation of Images and Videoscited by 99
- 2021DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Camerascited by 67
- 2022CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentationcited by 60
- 2022Waymo Open Dataset: Panoramic Video Panoptic Segmentationcited by 58
- 2021Part-aware Panoptic Segmentationcited by 56
- 2021Cross-View Regularization for Domain Adaptive Panoptic Segmentationcited by 41
- 2021Exemplar-Based Open-Set Panoptic Segmentation Networkcited by 38
38 more
- 2022Panoptic-PartFormer: Learning a Unified Model for Panoptic Part Segmentationcited by 38
- 2021PolyphonicFormer: Unified Query Learning for Depth-aware Video Panoptic Segmentationcited by 36
- 2022TubeFormer-DeepLab: Video Mask Transformercited by 35
- 2021SOLO: A Simple Framework for Instance Segmentationcited by 32
- 2023CLUSTSEG: Clustering for Universal Segmentationcited by 32
- 2021Fully Convolutional Networks for Panoptic Segmentation with Point-based Supervisioncited by 31
- 2023Semantic Image Segmentation: Two Decades of Researchcited by 27
- 2023Video-kMaX: A Simple Unified Approach for Online and Near-Online Video Panoptic Segmentationcited by 24
- 2021DeepLab2: A TensorFlow Library for Deep Labelingcited by 22
- 2024MG-LLaVA: Towards Multi-Granularity Visual Instruction Tuningcited by 22
- 2023SANeRF-HQ: Segment Anything for NeRF in High Qualitycited by 20
- 2025PanSt3R: Multi-view Consistent Panoptic Segmentationcited by 20
- 2023PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentationcited by 19
- 2023DFormer: Diffusion-guided Transformer for Universal Image Segmentationcited by 19
- 2021Panoptic Segmentation: A Reviewcited by 18
- 2023ReMaX: Relaxing for Better Training on Efficient Panoptic Segmentationcited by 18
- 2022Interactive Image Synthesis with Panoptic Layout Generationcited by 17
- 2022Uncertainty-aware Panoptic Segmentationcited by 17
- 2023PUPS: Point Cloud Unified Panoptic Segmentationcited by 16
- 2025Vision-Language Embodiment for Monocular Depth Estimationcited by 16
- 2022Backbones-Review: Feature Extraction Networks for Deep Learning and Deep Reinforcement Learning Approachescited by 15
- 2023RayMVSNet++: Learning Ray-based 1D Implicit Fields for Accurate Multi-View Stereocited by 15
- 2022PanopticDepth: A Unified Framework for Depth-aware Panoptic Segmentationcited by 14
- 2022Panoramic Panoptic Segmentation: Insights Into Surrounding Parsing for Mobile Agents via Unsupervised Contrastive Learningcited by 14
- 2022kMaX-DeepLab: k-means Mask Transformercited by 12
- 2021SMAC-Seg: LiDAR Panoptic Segmentation via Sparse Multi-directional Attention Clusteringcited by 11
- 2022LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Networkcited by 11
- 2022SFNet: Faster and Accurate Semantic Segmentation via Semantic Flowcited by 10
- 2022Visual Recognition by Requestcited by 10
- 2022Panoptic, Instance and Semantic Relations: A Relational Context Encoder to Enhance Panoptic Segmentationcited by 9
- 2022Dual Decision Improves Open-Set Panoptic Segmentationcited by 9
- 2022UniDAformer: Unified Domain Adaptive Panoptic Segmentation Transformer via Hierarchical Mask Calibrationcited by 8
- 2021Panoptic Segmentation Forecastingcited by 7
- 2022A Survey of Computer Vision Technologies In Urban and Controlled-environment Agriculturecited by 6
- 2023Dual Residual Attention Network for Image Denoisingcited by 6
- 2023Towards Deeply Unified Depth-aware Panoptic Segmentation with Bi-directional Guidance Learningcited by 6
- 2021Graphonomy: Universal Image Parsing via Graph Reasoning and Transfercited by 5
- 2021Panoptic Lintention Network: Towards Efficient Navigational Perception for the Visually Impairedcited by 5
50 most cited of 71 All 71 (JSON)
- 2026MambaPanoptic: A Vision Mamba-based Structured State Space Framework for Panoptic Segmentation12 May 2026
- 2025PanSt3R: Multi-view Consistent Panoptic Segmentation26 Jun 2025
- 2025Point or Line? Using Line-based Representation for Panoptic Symbol Spotting in CAD Drawings29 May 2025
- 2025A Data-Centric Approach to 3D Semantic Segmentation of Railway Scenes25 Apr 2025
- 2025Vision-Language Embodiment for Monocular Depth Estimation18 Mar 2025
- 2024CADSpotting: Robust Panoptic Symbol Spotting on Large-Scale CAD Drawings10 Dec 2024
- 2024Resolving Inconsistent Semantics in Multi-Dataset Image Segmentation15 Sep 2024
- 2024MG-LLaVA: Towards Multi-Granularity Visual Instruction Tuning25 Jun 2024
- 20242nd Place Solution for PVUW Challenge 2024: Video Panoptic Segmentation1 Jun 2024
- 2024Deep Instruction Tuning for Segment Anything Model31 Mar 2024
- 2024Depth-aware Panoptic Segmentation21 Mar 2024
- 2024HyCTAS: Multi-Objective Hybrid Convolution-Transformer Architecture Search for Real-Time Image Segmentation15 Mar 2024
38 more
- 2023EfficientPPS: Part-aware Panoptic Segmentation of Transparent Objects for Robotic Manipulation21 Dec 2023
- 2023SANeRF-HQ: Segment Anything for NeRF in High Quality3 Dec 2023
- 2023Towards Deeply Unified Depth-aware Panoptic Segmentation with Bi-directional Guidance Learning27 Jul 2023
- 2023RayMVSNet++: Learning Ray-based 1D Implicit Fields for Accurate Multi-View Stereo16 Jul 2023
- 2023ReMaX: Relaxing for Better Training on Efficient Panoptic Segmentation29 Jun 2023
- 2023Improving Panoptic Segmentation for Nighttime or Low-Illumination Urban Driving Scenes23 Jun 2023
- 2023DFormer: Diffusion-guided Transformer for Universal Image Segmentation6 Jun 2023
- 2023Dual Residual Attention Network for Image Denoising7 May 2023
- 2023Personalize Segment Anything Model with One Shot4 May 2023
- 2023CLUSTSEG: Clustering for Universal Segmentation3 May 2023
- 2023Intra-Batch Supervision for Panoptic Segmentation on High-Resolution Images17 Apr 2023
- 2023Video-kMaX: A Simple Unified Approach for Online and Near-Online Video Panoptic Segmentation10 Apr 2023
- 2023DCANet: Dual Convolutional Neural Network with Attention for Image Blind Denoising4 Apr 2023
- 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention28 Mar 2023
- 2023Semantic Image Segmentation: Two Decades of Research13 Feb 2023
- 2023PUPS: Point Cloud Unified Panoptic Segmentation13 Feb 2023
- 2023PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentation3 Jan 2023
- 2022PanDepth: Joint Panoptic Segmentation and Depth Completion29 Dec 2022
- 2022A Survey of Computer Vision Technologies In Urban and Controlled-environment Agriculture20 Oct 2022
- 2022A Generalist Framework for Panoptic Segmentation of Images and Videos12 Oct 2022
- 2022Time-Space Transformers for Video Panoptic Segmentation7 Oct 2022
- 2022SUNet: Scale-aware Unified Network for Panoptic Segmentation7 Sep 2022
- 2022Open-Vocabulary Universal Image Segmentation with MaskCLIP18 Aug 2022
- 2022Visual Recognition by Request28 Jul 2022
- 2022SFNet: Faster and Accurate Semantic Segmentation via Semantic Flow10 Jul 2022
- 2022kMaX-DeepLab: k-means Mask Transformer8 Jul 2022
- 2022Dual Decision Improves Open-Set Panoptic Segmentation6 Jul 2022
- 2022UniDAformer: Unified Domain Adaptive Panoptic Segmentation Transformer via Hierarchical Mask Calibration30 Jun 2022
- 2022Uncertainty-aware Panoptic Segmentation29 Jun 2022
- 2022Panoramic Panoptic Segmentation: Insights Into Surrounding Parsing for Mobile Agents via Unsupervised Contrastive Learning21 Jun 2022
- 2022CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation17 Jun 2022
- 2022Backbones-Review: Feature Extraction Networks for Deep Learning and Deep Reinforcement Learning Approaches16 Jun 2022
- 2022Waymo Open Dataset: Panoramic Video Panoptic Segmentation15 Jun 2022
- 2022PanopticDepth: A Unified Framework for Depth-aware Panoptic Segmentation1 Jun 2022
- 2022TubeFormer-DeepLab: Video Mask Transformer30 May 2022
- 2022Panoptic, Instance and Semantic Relations: A Relational Context Encoder to Enhance Panoptic Segmentation11 Apr 2022
- 2022Panoptic-PartFormer: Learning a Unified Model for Panoptic Part Segmentation10 Apr 2022
- 2022LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network14 Mar 2022
50 newest of 71 All 71 (JSON)
- 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
- 2023Personalize Segment Anything Model with One Shotcited by 250
- 2022Open-Vocabulary Universal Image Segmentation with MaskCLIPcited by 132
- 2021Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformerscited by 109
- 2021Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentationcited by 107
- 2022A Generalist Framework for Panoptic Segmentation of Images and Videoscited by 99
- 2021DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Camerascited by 67
- 2022CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentationcited by 60
- 2022Waymo Open Dataset: Panoramic Video Panoptic Segmentationcited by 58
- 2021Part-aware Panoptic Segmentationcited by 56
- 2021Cross-View Regularization for Domain Adaptive Panoptic Segmentationcited by 41
- 2021Exemplar-Based Open-Set Panoptic Segmentation Networkcited by 38
12 most cited of 69 All 69 (JSON)
- 2025PanSt3R: Multi-view Consistent Panoptic Segmentation26 Jun 2025
- 2025Point or Line? Using Line-based Representation for Panoptic Symbol Spotting in CAD Drawings29 May 2025
- 2025A Data-Centric Approach to 3D Semantic Segmentation of Railway Scenes25 Apr 2025
- 2025Vision-Language Embodiment for Monocular Depth Estimation18 Mar 2025
- 2024CADSpotting: Robust Panoptic Symbol Spotting on Large-Scale CAD Drawings10 Dec 2024
- 2024Resolving Inconsistent Semantics in Multi-Dataset Image Segmentation15 Sep 2024
- 20242nd Place Solution for PVUW Challenge 2024: Video Panoptic Segmentation1 Jun 2024
- 2024Deep Instruction Tuning for Segment Anything Model31 Mar 2024
- 2024Depth-aware Panoptic Segmentation21 Mar 2024
- 2024HyCTAS: Multi-Objective Hybrid Convolution-Transformer Architecture Search for Real-Time Image Segmentation15 Mar 2024
- 2023EfficientPPS: Part-aware Panoptic Segmentation of Transparent Objects for Robotic Manipulation21 Dec 2023
- 2023SANeRF-HQ: Segment Anything for NeRF in High Quality3 Dec 2023
12 newest of 69 All 69 (JSON)