mlsanityData to 7 Oct 2026

PartGLEE: A Foundation Model for Recognizing and Parsing Any Objects

arXiv 2407.16696 ↗First posted 23 Jul 2024cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

We present PartGLEE, a part-level foundation model for locating and identifying both objects and parts in images. Through a unified framework, PartGLEE accomplishes detection, segmentation, and grounding of instances at any granularity in the open world scenario. Specifically, we propose a Q-Former to construct the hierarchical relationship between objects and parts, parsing every object into corresponding semantic parts. By incorporating a large amount of object-level data, the hierarchical relationships can be extended, enabling PartGLEE to recognize a rich variety of parts. We conduct comprehensive studies to validate the effectiveness of our method, PartGLEE achieves the state-of-the-art performance across various part-level tasks and obtain competitive results on object-level tasks. The proposed PartGLEE significantly enhances hierarchical modeling capabilities and part-level perception over our previous GLEE model. Further analysis indicates that the hierarchical cognitive ability of PartGLEE is able to facilitate a detailed comprehension in images for mLLMs. The model and code will be released at https://provencestar.github.io/PartGLEE-Vision/ .

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 91 of its 105 references that are in the map. First posted on arXiv 23 Jul 2024.

Builds on

91 of 105 references.

References · 91

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  9. 2023Segment Anythingcited by 8,158
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  11. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  12. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
38 more
  1. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  3. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  4. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  6. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  8. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  9. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  10. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  11. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
  12. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  13. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  14. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  15. 2021Grounded Language-Image Pre-trainingcited by 1,147
  16. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  17. 2016Modeling Context in Referring Expressionscited by 1,054
  18. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  19. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  20. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  21. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  22. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  23. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  24. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  25. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  26. 2023Segment Everything Everywhere All at Oncecited by 546
  27. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
  28. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  29. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 420
  30. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
  31. 2019Video Instance Segmentationcited by 391
  32. 2023EVA-02: A Visual Representation for Neon Genesiscited by 377
  33. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 371
  34. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 341
  35. 2023Side Adapter Network for Open-Vocabulary Semantic Segmentationcited by 296
  36. 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
  37. 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 291
  38. 2022GLIPv2: Unifying Localization and Vision-Language Understandingcited by 275

50 most cited of 91 All 91 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 10; 2026: 4.

15 papers.

Sort:

By area:

  1. 2025Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priorscited by 9
  2. 20254D-Animal: Freely Reconstructing Animatable 3D Animals from Videoscited by 6
  3. 2024CALICO: Part-Focused Semantic Co-Segmentation with Large Vision-Language Modelscited by 4
  4. 2025Fine-Grained Image-Text Correspondence with Cost Aggregation for Open-Vocabulary Part Segmentationcited by 4
  5. 2026PCA-Seg: Revisiting Cost Aggregation for Open-Vocabulary Semantic and Part Segmentationcited by 4
  6. 2025Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Reviewcited by 3
  7. 2025LangHOPS: Language Grounded Hierarchical Open-Vocabulary Part Segmentationcited by 1
  8. 2025Deeply Dual Supervised learning for melanoma recognitioncited by 0
  9. 2025Edge Detection for Organ Boundaries via Top Down Refinement and SubPixel Upsamplingcited by 0
  10. 2025DualResolution Residual Architecture with Artifact Suppression for Melanocytic Lesion Segmentationcited by 0
  11. 2025VesselRW: Weakly Supervised Subcutaneous Vessel Segmentation via Learned Random Walk Propagationcited by 0
  12. 2025VisKnow: Constructing Visual Knowledge Base for Object Understandingcited by 0
3 more
  1. 2026SceneParser: Hierarchical Scene Parsing for Visual Semantics Understandingcited by 0
  2. 2026Depth-Dominant Skeleton Detection for Natural Scenescited by 0
  3. 2026SPK: Eliciting Structured Prior Knowledge for Interpretable Out-of-Distribution Detection in Real-Time Object Detectioncited by 0