mlsanityData to 14 Oct 2026

Hierarchical Collaborative Fusion for 3D Instance-aware Referring Expression Segmentation

arXiv 2603.06250 ↗First posted 6 Mar 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Generalised 3D Referring Expression Segmentation (3D-GRES) localizes objects in 3D scenes based on natural language, even when descriptions match multiple or zero targets. Existing methods rely solely on sparse point clouds, lacking rich visual semantics for fine-grained descriptions. We propose HCF-RES, a multi-modal framework with two key innovations. First, Hierarchical Visual Semantic Decomposition leverages SAM instance masks to guide CLIP encoding at dual granularities -- pixel-level and instance-level features -- preserving object boundaries during 2D-to-3D projection. Second, Progressive Multi-level Fusion integrates representations through intra-modal collaboration, cross-modal adaptive weighting between 2D semantic and 3D geometric features, and language-guided refinement. HCF-RES achieves state-of-the-art results on both ScanRefer and Multi3DRefer.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 26 references that are in the map. First posted on arXiv 6 Mar 2026.

Builds on

24 of 26 references.

References · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
  4. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 473
  5. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
  6. 2017Large-scale Point Cloud Semantic Segmentation with Superpoint Graphscited by 385
  7. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  8. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  9. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 146
  10. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 141
  11. 2022Superpoint Transformer for 3D Scene Instance Segmentationcited by 116
  12. 2022EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Groundingcited by 99
12 more
  1. 2024IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detectioncited by 57
  2. 20233D-STMN: Dependency-Driven Superpoint-Text Matching Network for End-to-End 3D Referring Expression Segmentationcited by 36
  3. 2024SegPoint: Segment Any Point Cloud via Large Language Modelcited by 35
  4. 2023Four Ways to Improve Verbo-visual Fusion for Dense 3D Visual Groundingcited by 32
  5. 2024Reason3D: Searching and Reasoning 3D Segmentation via Large Language Modelcited by 28
  6. 2024RefMask3D: Language-Guided Transformer for 3D Referring Segmentationcited by 25
  7. 2024Multi-branch Collaborative Learning Network for 3D Visual Groundingcited by 20
  8. 20243D-GRES: Generalized 3D Referring Expression Segmentationcited by 18
  9. 2025MLLM-For3D: Adapting Multimodal Large Language Model for 3D Reasoning Segmentationcited by 16
  10. 2024RG-SAN: Rule-Guided Spatial Awareness Network for End-to-End 3D Referring Expression Segmentationcited by 11
  11. 2025IPDN: Image-enhanced Prompt Decoding Network for 3D Referring Expression Segmentationcited by 8
  12. 2023A Unified Framework for 3D Point Cloud Visual Groundingcited by 6

Built on by

Not cited yet.