Simultaneous Detection and Segmentation
Not on the map. Why?
Abstract (arXiv)
We aim to detect all instances of a category in an image and, for each instance, mark the pixels that belong to it. We call this task Simultaneous Detection and Segmentation (SDS). Unlike classical bounding box detection, SDS requires a segmentation and not just a box. Unlike classical semantic segmentation, we require individual object instances. We build on recent work that uses convolutional neural networks to classify category-independent region proposals (R-CNN [16]), introducing a novel architecture tailored for SDS. We then use category-specific, top- down figure-ground predictions to refine our bottom-up proposals. We show a 7 point boost (16% relative) over our baselines on SDS, a 5 point boost (10% relative) over state-of-the-art on semantic segmentation, and state-of-the-art performance in object detection. Finally, we provide diagnostic tools that unpack performance and provide directions for future work.
Key facts. Cited by 158 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jul 2014.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 41; 2022: 46; 2023: 29; 2024: 20; 2025: 15; 2026: 6.
158 papers.
Sort:
By area:
- 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
- 2021Bottleneck Transformers for Visual Recognitioncited by 364
- 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 327
- 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIPcited by 199
- 2021Hypercorrelation Squeeze for Few-Shot Segmentationcited by 193
- 2021Few-Shot Segmentation via Cycle-Consistent Transformercited by 133
- 2021Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformercited by 116
- 2022Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentationcited by 114
- 2023CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Predictioncited by 114
- 2021Self-Guided and Cross-Guided Learning for Few-Shot Segmentationcited by 92
- 2021Weakly Supervised Object Localization and Detection: A Surveycited by 84
- 2021STEP: Segmenting and Tracking Every Pixelcited by 72
38 more
- 2024The Ingredients for Robotic Diffusion Transformerscited by 72
- 2022RbA: Segmenting Unknown Regions Rejected by Allcited by 67
- 2022Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentationcited by 65
- 2022CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentationcited by 60
- 2021DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervisioncited by 59
- 2022Feature-Proxy Transformer for Few-Shot Segmentationcited by 58
- 2022Waymo Open Dataset: Panoramic Video Panoptic Segmentationcited by 57
- 2021Learning Meta-class Memory for Few-Shot Semantic Segmentationcited by 56
- 2021ISTR: End-to-End Instance Segmentation with Transformerscited by 52
- 2021Affinity Attention Graph Neural Network for Weakly Supervised Semantic Segmentationcited by 47
- 2022MOAT: Alternating Mobile Convolution and Attention Brings Strong Vision Modelscited by 47
- 2022A Survey on Label-efficient Deep Image Segmentation: Bridging the Gap between Weak Supervision and Dense Predictioncited by 46
- 2021Deep Reinforcement Learning in Computer Vision: A Comprehensive Surveycited by 42
- 2023On Advantages of Mask-level Recognition for Outlier-aware Segmentationcited by 42
- 2021Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervisioncited by 39
- 2021Few-shot Segmentation with Optimal Transport Matching and Message Flowcited by 36
- 2022Learning Pixel Trajectories with Multiscale Contrastive Random Walkscited by 36
- 2022BlobGAN: Spatially Disentangled Scene Representationscited by 35
- 2022TubeFormer-DeepLab: Video Mask Transformercited by 35
- 2023Loss Functions in the Era of Semantic Segmentation: A Survey and Outlookcited by 33
- 2021SOLO: A Simple Framework for Instance Segmentationcited by 32
- 2023Self-Calibrated Cross Attention Network for Few-Shot Segmentationcited by 32
- 2021Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsingcited by 31
- 2023DaTaSeg: Taming a Universal Multi-Dataset Multi-Task Segmentation Modelcited by 30
- 2023Semantic Image Segmentation: Two Decades of Researchcited by 26
- 2022Interclass Prototype Relation for Few-Shot Segmentationcited by 25
- 2023MIANet: Aggregating Unbiased Instance and General Information for Few-Shot Semantic Segmentationcited by 25
- 2024COCONut: Modernizing COCO Segmentationcited by 24
- 2022Doubly Deformable Aggregation of Covariance Matrices for Few-shot Segmentationcited by 23
- 2021DeepLab2: A TensorFlow Library for Deep Labelingcited by 22
- 2022A Strong Baseline for Generalized Few-Shot Semantic Segmentationcited by 22
- 2022Mask Matching Transformer for Few-Shot Segmentationcited by 22
- 2023FECANet: Boosting Few-Shot Semantic Segmentation with Feature-Enhanced Context-Aware Networkcited by 22
- 2023Object Detection with Transformers: A Reviewcited by 22
- 2024Hybrid Mamba for Few-Shot Segmentationcited by 21
- 2024Eliminating Feature Ambiguity for Few-Shot Segmentationcited by 19
- 2023Deep Learning Technique for Human Parsing: A Survey and Outlookcited by 18
- 2023ReMaX: Relaxing for Better Training on Efficient Panoptic Segmentationcited by 18
50 most cited of 158 All 158 (JSON)
- 2026Exploiting Target Knowledge from MLLMs for Robust Few-Shot Segmentation24 Sep 2026
- 2026CoDS: Robust Collaborative Perception via Expert-driven Detection and BEV Segmentation14 Aug 2026
- 2026Redefining Instance Matching: A Unified Framework for Part-Aware Matching in Panoptic Segmentation Evaluation29 May 2026
- 2026From Pixels to Concepts: Do Segmentation Models Understand What They Segment?10 May 2026
- 2026Unify the Views: View-Consistent Prototype Learning for Few-Shot Segmentation6 Mar 2026
- 2026ReScene4D: Temporally Consistent Semantic Instance Segmentation of Evolving Indoor 3D Scenes16 Jan 2026
- 2025What Holds Back Open-Vocabulary Segmentation?6 Aug 2025
- 2025Rethinking Query-based Transformer for Continual Image Segmentation10 Jul 2025
- 2025What Demands Attention in Urban Street Scenes? From Scene Understanding towards Road Safety: A Survey of Vision-driven Datasets and Studies9 Jul 2025
- 2025Inter- and Intra-image Refinement for Few Shot Segmentation8 Jul 2025
- 2025Unlocking Constraints: Source-Free Occlusion-Aware Seamless Segmentation26 Jun 2025
- 2025IPFormer: Visual 3D Panoptic Scene Completion with Context-Adaptive Instance Proposals25 Jun 2025
38 more
- 2025Unlocking the Power of SAM 2 for Few-Shot Segmentation20 May 2025
- 2025Vision-Language Model for Object Detection and Segmentation: A Review and Evaluation13 Apr 2025
- 2025MARS: a Multimodal Alignment and Ranking System for Few-Shot Segmentation10 Apr 2025
- 2025SydneyScapes: Image Segmentation for Australian Environments10 Apr 2025
- 2025Benchmarking Object Detectors under Real-World Distribution Shifts in Satellite Imagery24 Mar 2025
- 2025LEGO-Motion: Learning-Enhanced Grids with Occupancy Instance Modeling for Class-Agnostic Motion Prediction10 Mar 2025
- 2025Temporal-consistent CAMs for Weakly Supervised Video Segmentation in Waste Sorting3 Feb 2025
- 2025Beyond-Labels: Advancing Open-Vocabulary Segmentation With Vision-Language Models28 Jan 2025
- 2025Foreground-Covering Prototype Generation and Matching for SAM-Aided Few-Shot Segmentation1 Jan 2025
- 2024A Novel Shape Guided Transformer Network for Instance Segmentation in Remote Sensing Images31 Dec 2024
- 2024FCC: Fully Connected Correlation for One-Shot Segmentation18 Nov 2024
- 2024Configurable Embodied Data Generation for Class-Agnostic RGB-D Video Segmentation16 Oct 2024
- 2024The Ingredients for Robotic Diffusion Transformers14 Oct 2024
- 2024DMC-Net: Lightweight Dynamic Multi-Scale and Multi-Resolution Convolution Network for Pancreas Segmentation in CT Images3 Oct 2024
- 2024Hybrid Mamba for Few-Shot Segmentation29 Sep 2024
- 2024AgMTR: Agent Mining Transformer for Few-shot Segmentation in Remote Sensing26 Sep 2024
- 2024High-Performance Few-Shot Segmentation with Foundation Models: An Empirical Study10 Sep 2024
- 2024Beyond Mask: Rethinking Guidance Types in Few-shot Segmentation16 Jul 2024
- 2024Eliminating Feature Ambiguity for Few-Shot Segmentation13 Jul 2024
- 2024Occlusion-Aware Seamless Segmentation2 Jul 2024
- 2024OoDIS: Anomaly Instance Segmentation and Detection Benchmark17 Jun 2024
- 2024PanoSSC: Exploring Monocular Panoptic 3D Scene Reconstruction for Autonomous Driving11 Jun 2024
- 2024MP-PolarMask: A Faster and Finer Instance Segmentation for Concave Images3 Jun 2024
- 2024P-MSDiff: Parallel Multi-Scale Diffusion for Remote Sensing Image Segmentation30 May 2024
- 2024Learning Robust Correlation with Foundation Model for Weakly-Supervised Few-Shot Segmentation30 May 2024
- 2024COCONut: Modernizing COCO Segmentation12 Apr 2024
- 2024HAPNet: Toward Superior RGB-Thermal Scene Parsing via Hybrid, Asymmetric, and Progressive Heterogeneous Feature Fusion4 Apr 2024
- 2024SISP: A Benchmark Dataset for Fine-grained Ship Instance Segmentation in Panchromatic Satellite Images6 Feb 2024
- 2024Boosting Few-Shot Semantic Segmentation Via Segment Anything Model18 Jan 2024
- 2023Task-Disruptive Background Suppression for Few-Shot Segmentation26 Dec 2023
- 2023Adaptive FSS: A Novel Few-Shot Segmentation Framework via Prototype Enhancement25 Dec 2023
- 2023MaskConver: Revisiting Pure Convolution Model for Panoptic Segmentation11 Dec 2023
- 2023Loss Functions in the Era of Semantic Segmentation: A Survey and Outlook8 Dec 2023
- 2023JPPF: Multi-task Fusion for Consistent Panoptic-Part Segmentation30 Nov 2023
- 2023Focus on Query: Adversarial Mining Transformer for Few-Shot Segmentation29 Nov 2023
- 2023Continual Learning for Image Segmentation with Dynamic Query29 Nov 2023
- 2023PolyMaX: General Dense Prediction with Mask Transformer9 Nov 2023
- 2023CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Prediction2 Oct 2023
50 newest of 158 All 158 (JSON)
- 2024Hybrid Mamba for Few-Shot Segmentationcited by 21
- 2024Hybrid Mamba for Few-Shot Segmentation29 Sep 2024
- 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
- 2021Bottleneck Transformers for Visual Recognitioncited by 364
- 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 327
- 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIPcited by 199
- 2021Hypercorrelation Squeeze for Few-Shot Segmentationcited by 193
- 2021Few-Shot Segmentation via Cycle-Consistent Transformercited by 133
- 2021Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformercited by 116
- 2022Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentationcited by 114
- 2023CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Predictioncited by 114
- 2021Self-Guided and Cross-Guided Learning for Few-Shot Segmentationcited by 92
- 2021Weakly Supervised Object Localization and Detection: A Surveycited by 84
- 2021STEP: Segmenting and Tracking Every Pixelcited by 72
12 most cited of 150 All 150 (JSON)
- 2026Exploiting Target Knowledge from MLLMs for Robust Few-Shot Segmentation24 Sep 2026
- 2026Redefining Instance Matching: A Unified Framework for Part-Aware Matching in Panoptic Segmentation Evaluation29 May 2026
- 2026From Pixels to Concepts: Do Segmentation Models Understand What They Segment?10 May 2026
- 2026Unify the Views: View-Consistent Prototype Learning for Few-Shot Segmentation6 Mar 2026
- 2026ReScene4D: Temporally Consistent Semantic Instance Segmentation of Evolving Indoor 3D Scenes16 Jan 2026
- 2025What Holds Back Open-Vocabulary Segmentation?6 Aug 2025
- 2025Rethinking Query-based Transformer for Continual Image Segmentation10 Jul 2025
- 2025What Demands Attention in Urban Street Scenes? From Scene Understanding towards Road Safety: A Survey of Vision-driven Datasets and Studies9 Jul 2025
- 2025Inter- and Intra-image Refinement for Few Shot Segmentation8 Jul 2025
- 2025Unlocking Constraints: Source-Free Occlusion-Aware Seamless Segmentation26 Jun 2025
- 2025IPFormer: Visual 3D Panoptic Scene Completion with Context-Adaptive Instance Proposals25 Jun 2025
- 2025Unlocking the Power of SAM 2 for Few-Shot Segmentation20 May 2025
12 newest of 150 All 150 (JSON)