mlsanityData to 7 Oct 2026

LangHOPS: Language Grounded Hierarchical Open-Vocabulary Part Segmentation

arXiv 2510.25263 ↗First posted 29 Oct 2025cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

We propose LangHOPS, the first Multimodal Large Language Model (MLLM) based framework for open-vocabulary object-part instance segmentation. Given an image, LangHOPS can jointly detect and segment hierarchical object and part instances from open-vocabulary candidate categories. Unlike prior approaches that rely on heuristic or learnable visual grouping, our approach grounds object-part hierarchies in language space. It integrates the MLLM into the object-part parsing pipeline to leverage its rich knowledge and reasoning capabilities, and link multi-granularity concepts within the hierarchies. We evaluate LangHOPS across multiple challenging scenarios, including in-domain and cross-dataset object-part instance segmentation, and zero-shot semantic segmentation. LangHOPS achieves state-of-the-art results, surpassing previous methods by 5.5% Average Precision (AP) (in-domain) and 4.8% (cross-dataset) on the PartImageNet dataset and by 2.5% mIOU on unseen object parts in ADE20K (zero-shot). Ablation studies further validate the effectiveness of the language-grounded hierarchy and MLLM driven part query refinement strategy. The code will be released here.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 49 of its 66 references that are in the map. First posted on arXiv 29 Oct 2025.

Builds on

49 of 66 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Real Classification by Description: Extending CLIP's Limits of Part Attributes Recognition18 Dec

Other references · 48

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2023Segment Anythingcited by 8,158
  4. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  5. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  6. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  7. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  8. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  9. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  10. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  11. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  12. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
36 more
  1. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  2. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 560
  3. 2023Segment Everything Everywhere All at Oncecited by 546
  4. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
  5. 2023LangSplat: 3D Language Gaussian Splattingcited by 410
  6. 2014Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Partscited by 230
  7. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 171
  8. 2023PACO: Parts and Attributes of Common Objectscited by 146
  9. 2022Open-Vocabulary Universal Image Segmentation with MaskCLIPcited by 131
  10. 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understandingcited by 126
  11. 2021PartImageNet: A Large, High-Quality Dataset of Partscited by 107
  12. 2024URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Imagescited by 106
  13. 2024PSALM: Pixelwise SegmentAtion with Large Multi-Modal Modelcited by 100
  14. 2024Splat-SLAM: Globally Optimized RGB-only SLAM with 3D Gaussianscited by 64
  15. 2024EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Modelcited by 63
  16. 2021Part-aware Panoptic Segmentationcited by 56
  17. 2023Going Denser with Open-Vocabulary Part Segmentationcited by 51
  18. 2022Masked Jigsaw Puzzle: A Versatile Position Embedding for Vision Transformerscited by 33
  19. 2024HyperSeg: Towards Universal Visual Segmentation with Large Language Modelcited by 33
  20. 2023OV-PARTS: Towards Open-Vocabulary Part Segmentationcited by 31
  21. 2025OmniSAM: Omnidirectional Segment Anything Model for UDA in Panoramic Semantic Segmentationcited by 26
  22. 2022When are Lemons Purple? The Concept Association Bias of Vision-Language Modelscited by 23
  23. 2023PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentationcited by 19
  24. 2023Towards Open-World Segmentation of Partscited by 15
  25. 2024PartGLEE: A Foundation Model for Recognizing and Parsing Any Objectscited by 15
  26. 2023Direction-Oriented Visual-semantic Embedding Model for Remote Sensing Image-text Retrievalcited by 14
  27. 2024CoARF: Controllable 3D Artistic Style Transfer for Radiance Fieldscited by 14
  28. 2024Sharing Key Semantics in Transformer Makes Efficient Image Restorationcited by 14
  29. 2025CamSAM2: Segment Anything Accurately in Camouflaged Videoscited by 7
  30. 2024Task-aligned Part-aware Panoptic Segmentation through Joint Object-Part Representationscited by 4
  31. 2024CALICO: Part-Focused Semantic Co-Segmentation with Large Vision-Language Modelscited by 4
  32. 2025Fine-Grained Image-Text Correspondence with Cost Aggregation for Open-Vocabulary Part Segmentationcited by 4
  33. 2024RISE-SDF: a Relightable Information-Shared Signed Distance Field for Glossy Object Inverse Renderingcited by 3
  34. 2025Unlocking Constraints: Source-Free Occlusion-Aware Seamless Segmentationcited by 3
  35. 2023JPPF: Multi-task Fusion for Consistent Panoptic-Part Segmentationcited by 2
  36. 2025Split Matching for Inductive Zero-shot Semantic Segmentationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026The Eleventh NTIRE 2026 Efficient Super-Resolution Challenge Reportcited by 25

Similar, not cited

Close in wording, no citation either way

  1. 2024Transferable and Principled Efficiency for Open-Vocabulary Segmentation
  2. 2025Leveraging Vision-Language Models for Open-Vocabulary Instance Segmentation and Tracking
  3. 2023Diffusion Models for Open-Vocabulary Segmentation
  4. 2025From Open-Vocabulary to Vocabulary-Free Semantic Segmentation
  5. 2025Stepping Out of Similar Semantic Space for Open-Vocabulary Segmentation
  6. 2025Vision-Language Model for Object Detection and Segmentation: A Review and Evaluation
  7. 20263D-PLOT-LLM: Part-Level Object Tokens for 3D Large Language Models
  8. 2023Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models