mlsanityData to 7 Oct 2026

Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity

arXiv 2204.06107 ↗First posted 12 Apr 2022cs.CVcited by 41

Not on the map. Why?

Abstract (arXiv)

Open-world instance segmentation is the task of grouping pixels into object instances without any pre-determined taxonomy. This is challenging, as state-of-the-art methods rely on explicit class semantics obtained from large labeled datasets, and out-of-domain evaluation performance drops significantly. Here we propose a novel approach for mask proposals, Generic Grouping Networks (GGNs), constructed without semantic supervision. Our approach combines a local measure of pixel affinity with instance-level mask supervision, producing a training regimen designed to make the model as generic as the data diversity allows. We introduce a method for predicting Pairwise Affinities (PA), a learned local relationship between pairs of pixels. PA generalizes very well to unseen categories. From PA we construct a large set of pseudo-ground-truth instance masks; combined with human-annotated instance masks we train GGNs and significantly outperform the SOTA on open-world instance segmentation on various benchmarks including COCO, LVIS, ADE20K, and UVO. Code is available on project website: https://sites.google.com/view/generic-grouping/.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Builds on 40 of its 66 references that are in the map. First posted on arXiv 12 Apr 2022.

Builds on

40 of 66 references.

References · 40

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2017Mask R-CNNcited by 7,259
  7. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  8. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  9. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  10. 2017The Kinetics Human Action Video Datasetcited by 1,894
  11. 2018Group Normalizationcited by 1,514
  12. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
28 more
  1. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  2. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  3. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  4. 2018Panoptic Segmentationcited by 796
  5. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  6. 2015Holistically-Nested Edge Detectioncited by 590
  7. 2018Rethinking ImageNet Pre-trainingcited by 376
  8. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  9. 2020Rethinking Pre-training and Self-trainingcited by 352
  10. 2021Towards Open World Object Detectioncited by 297
  11. 2020A Simple Semi-Supervised Learning Framework for Object Detectioncited by 282
  12. 2014Towards Open World Recognitioncited by 226
  13. 2019Learning Data Augmentation Strategies for Object Detectioncited by 139
  14. 2021Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentationcited by 133
  15. 2019Learning the Depths of Moving People by Watching Frozen Peoplecited by 120
  16. 2021Learning Open-World Object Proposals without Learning to Classifycited by 109
  17. 2019Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detectioncited by 104
  18. 2019SSAP: Single-Shot Instance Segmentation With Affinity Pyramidcited by 85
  19. 2015Learning to Segment Object Candidatescited by 79
  20. 2017Learning Affinity via Spatial Propagation Networkscited by 79
  21. 2020Self-Supervised Visual Representation Learning from Hierarchical Groupingcited by 63
  22. 2021Opening up Open-World Trackingcited by 50
  23. 2019Devil is in the Edges: Learning Semantic Boundaries from Noisy Annotationscited by 32
  24. 2018Affinity Derivation and Graph Merge for Instance Segmentationcited by 25
  25. 2018Semantic Edge Detection with Diverse Deep Supervisioncited by 13
  26. 2015Object-Proposal Evaluation Protocol is 'Gameable'cited by 9
  27. 2015Affinity CNN: Learning Pixel-Centric Pairwise Relations for Figure/Ground Embeddingcited by 9
  28. 2020What leads to generalization of object proposals?cited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 18; 2024: 8; 2025: 7; 2026: 3.

41 papers.

Sort:

By area:

  1. 2023Segment Anythingcited by 8,158
  2. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 371
  3. 2023Towards Open Vocabulary Learning: A Surveycited by 165
  4. 2022PROB: Probabilistic Objectness for Open World Object Detectioncited by 57
  5. 2023MobileSAMv2: Faster Segment Anything to Everythingcited by 41
  6. 2023EgoTracks: A Long-term Egocentric Visual Object Tracking Datasetcited by 27
  7. 2023A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Applicationcited by 21
  8. 2023Spatially Covariant Image Registration with Text Promptscited by 18
  9. 2023SegPrompt: Boosting Open-world Segmentation via Category-level Prompt Learningcited by 17
  10. 2023Hyp-OW: Exploiting Hierarchical Structure Learning with Hyperbolic Distance Enhances Open World Object Detectioncited by 14
  11. 2022GOOD: Exploring Geometric Cues for Detecting Objects in an Open Worldcited by 10
  12. 2024HASSOD: Hierarchical Adaptive Self-Supervised Object Detectioncited by 10
29 more
  1. 2024Generative Active Learning for Long-tailed Instance Segmentationcited by 10
  2. 2023Open-world Instance Segmentation: Top-down Learning with Bottom-up Supervisioncited by 8
  3. 20233D Indoor Instance Segmentation in an Open-Worldcited by 7
  4. 2024FAST-Splat: Fast, Ambiguity-Free Semantics Transfer in Gaussian Splattingcited by 7
  5. 2022Beyond mAP: Towards better evaluation of instance segmentationcited by 6
  6. 2023Unsupervised Recognition of Unknown Objects for Open-World Object Detectioncited by 6
  7. 2024SOHES: Self-supervised Open-world Hierarchical Entity Segmentationcited by 6
  8. 2023Video Instance Segmentation in an Open-Worldcited by 5
  9. 2023Exploring Transformers for Open-world Instance Segmentationcited by 5
  10. 2022Single-Stage Open-world Instance Segmentation with Cross-task Consistency Regularizationcited by 4
  11. 2022SupeRGB-D: Zero-shot Instance Segmentation in Cluttered Indoor Environmentscited by 4
  12. 2023OpenInst: A Simple Query-Based Method for Open-World Instance Segmentationcited by 4
  13. 2024ActionSwitch: Class-agnostic Detection of Simultaneous Actions in Streaming Videoscited by 4
  14. 2023ElC-OIS: Ellipsoidal Clustering for Open-World Instance Segmentation on LiDAR Datacited by 3
  15. 2024SOS: Segment Object System for Open-World Instance Segmentation With Object Priorscited by 3
  16. 2023Addressing the Challenges of Open-World Object Detectioncited by 2
  17. 2023Low-shot Object Learning with Mutual Exclusivity Biascited by 2
  18. 2024UADet: A Remarkably Simple Yet Effective Uncertainty-Aware Open-Set Object Detection Frameworkcited by 2
  19. 2025Generalized Class Discovery in Instance Segmentationcited by 2
  20. 2025Towards 3D Objectness Learning in an Open Worldcited by 2
  21. 2025v-CLR: View-Consistent Learning for Open-World Instance Segmentationcited by 1
  22. 2025Improving Open-World Object Localization by Discovering Backgroundcited by 1
  23. 2024Utilizing Grounded SAM for self-supervised frugal camouflaged human detectioncited by 0
  24. 2025Learning 3D Texture-Aware Representations for Parsing Diverse Human Clothing and Body Partscited by 0
  25. 2025Towards Open World Detection: A Surveycited by 0
  26. 2025Efficient Generalization via Multimodal Co-Training under Data Scarcity and Distribution Shiftcited by 0
  27. 2026DynFOA: Generating First-Order Ambisonics with Conditional Diffusion for Dynamic and Acoustically Complex 360-Degree Videoscited by 0
  28. 2026DynFOA: Generating First-Order Ambisonics with Conditional Diffusion for Dynamic and Acoustically Complex 360-Degree Videoscited by 0
  29. 2026Background-Free Objectness Learning for Class-Agnostic Detectioncited by 0