mlsanityData to 7 Oct 2026

SPIN: Hierarchical Segmentation with Subpart Granularity in Natural Images

arXiv 2407.09686 ↗First posted 12 Jul 2024cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Hierarchical segmentation entails creating segmentations at varying levels of granularity. We introduce the first hierarchical semantic segmentation dataset with subpart annotations for natural images, which we call SPIN (SubPartImageNet). We also introduce two novel evaluation metrics to evaluate how well algorithms capture spatial and semantic relationships across hierarchical levels. We benchmark modern models across three different tasks and analyze their strengths and weaknesses across objects, parts, and subparts. To facilitate community-wide progress, we publicly release our dataset at https://joshmyersdean.github.io/spin/index.html.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 61 of its 75 references that are in the map. First posted on arXiv 12 Jul 2024.

Builds on

61 of 75 references.

References · 61

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2023Segment Anythingcited by 8,158
  9. 2023Visual Instruction Tuningcited by 7,792
  10. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  11. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  12. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
38 more
  1. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
  2. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  3. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  4. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  5. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
  6. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 498
  7. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 447
  8. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  9. 2023EVA-02: A Visual Representation for Neon Genesiscited by 377
  10. 2014Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Partscited by 230
  11. 2023PixelLM: Pixel Reasoning with Large Multimodal Modelcited by 220
  12. 2020RoboTHOR: An Open Simulation-to-Real Embodied AI Platformcited by 190
  13. 2023Semantic-SAM: Segment and Recognize Anything at Any Granularitycited by 185
  14. 2019CvxNet: Learnable Convex Decompositioncited by 173
  15. 2023ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Promptscited by 160
  16. 2023Osprey: Pixel Understanding with Visual Instruction Tuningcited by 158
  17. 2024Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokenscited by 148
  18. 2023PACO: Parts and Attributes of Common Objectscited by 146
  19. 2022GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Partscited by 135
  20. 2018Instance-level Human Parsing via Part Grouping Networkcited by 134
  21. 2023Hyperbolic Image-Text Representationscited by 132
  22. 2021PartImageNet: A Large, High-Quality Dataset of Partscited by 107
  23. 2022Deep Hierarchical Semantic Segmentationcited by 84
  24. 2018Look into Person: Joint Body Parsing & Pose Estimation Network and A New Benchmarkcited by 79
  25. 2019A Logic-Driven Framework for Consistency of Neural Modelscited by 64
  26. 2018Understanding Humans in Crowded Scenes: Deep Nested Adversarial Learning and A New Benchmark for Multi-Human Parsingcited by 60
  27. 2018ApolloCar3D: A Large 3D Car Instance Understanding Benchmark for Autonomous Drivingcited by 59
  28. 2020Fashionpedia: Ontology, Segmentation, and an Attribute Localization Datasetcited by 57
  29. 2021Part-aware Panoptic Segmentationcited by 56
  30. 2023Hierarchical Open-vocabulary Universal Image Segmentationcited by 53
  31. 2023Going Denser with Open-Vocabulary Part Segmentationcited by 51
  32. 2016Semantic Object Parsing with Graph LSTMcited by 44
  33. 2019PartNet: A Recursive Part Decomposition Network for Fine-grained and Hierarchical Shape Segmentationcited by 44
  34. 2023See, Say, and Segment: Teaching LMMs to Overcome False Premisescited by 40
  35. 2022Panoptic-PartFormer: Learning a Unified Model for Panoptic Part Segmentationcited by 38
  36. 2021PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoningcited by 34
  37. 2018ModaNet: A Large-Scale Street Fashion Dataset with Polygon Annotationscited by 33
  38. 2023OV-PARTS: Towards Open-Vocabulary Part Segmentationcited by 31

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 1.

5 papers.

  1. 2024Analyzing The Language of Visual Tokenscited by 12
  2. 2025OVFact: Measuring and Improving Open-Vocabulary Factuality for Long Caption Modelscited by 2
  3. 2025PartStickers: Generating Parts of Objects for Rapid Prototypingcited by 0
  4. 2025Pancakes: Consistent Multi-Protocol Image Segmentation Across Biomedical Domainscited by 0
  5. 2026COCOTree: A Dataset and Benchmark for Open Tree-Structured Visual Decompositioncited by 0