# Multi-branch Collaborative Learning Network for 3D Visual Grounding

arXiv: https://arxiv.org/abs/2407.05363 · HTML: https://mlsanity.com/paper/2407.05363/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 48 references that are in the map.
- First posted on arXiv 7 Jul 2024.

## Builds on (most cited first, 30 of 32)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language](https://mlsanity.com/paper/1912.08830/) (2019)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval](https://mlsanity.com/paper/2207.07285/) (2022)
- [Group-Free 3D Object Detection via Transformers](https://mlsanity.com/paper/2104.00678/) (2021)
- [Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentation](https://mlsanity.com/paper/2003.08813/) (2020)
- [OccuSeg: Occupancy-aware 3D Instance Segmentation](https://mlsanity.com/paper/2003.06537/) (2020)
- [InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring](https://mlsanity.com/paper/2103.01128/) (2021)
- [Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Clouds](https://mlsanity.com/paper/2112.08879/) (2021)
- [SAT: 2D Semantics Assisted Training for 3D Visual Grounding](https://mlsanity.com/paper/2105.11450/) (2021)
- [Language Conditioned Spatial Relation Reasoning for 3D Object Grounding](https://mlsanity.com/paper/2211.09646/) (2022)
- [Superpoint Transformer for 3D Scene Instance Segmentation](https://mlsanity.com/paper/2211.15766/) (2022)
- [3D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selection](https://mlsanity.com/paper/2204.06272/) (2022)
- [Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks](https://mlsanity.com/paper/2108.07478/) (2021)
- [EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Grounding](https://mlsanity.com/paper/2209.14941/) (2022)
- [LanguageRefer: Spatial-Language Model for 3D Visual Grounding](https://mlsanity.com/paper/2107.03438/) (2021)
- [TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Grounding](https://mlsanity.com/paper/2108.02388/) (2021)
- [Free-form Description Guided 3D Visual Graph Network for Object Grounding in Point Cloud](https://mlsanity.com/paper/2103.16381/) (2021)
- [Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotations](https://mlsanity.com/paper/1809.04766/) (2018)
- [Look Around and Refer: 2D Synthetic Semantics Knowledge Distillation for 3D Visual Grounding](https://mlsanity.com/paper/2211.14241/) (2022)
- [X-Mesh: Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidance](https://mlsanity.com/paper/2303.15764/) (2023)
- [3D-STMN: Dependency-Driven Superpoint-Text Matching Network for End-to-End 3D Referring Expression Segmentation](https://mlsanity.com/paper/2308.16632/) (2023)
- [Towards Local Visual Modeling for Image Captioning](https://mlsanity.com/paper/2302.06098/) (2023)
- [RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free](https://mlsanity.com/paper/1901.03353/) (2019)
- [SAM as the Guide: Mastering Pseudo-Label Refinement in Semi-Supervised Referring Expression Segmentation](https://mlsanity.com/paper/2406.01451/) (2024)

## Built on by (most cited first, 20 of 20)

- [SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Grounding](https://mlsanity.com/paper/2412.04383/) (2024)
- [Multimodal Referring Segmentation: A Survey](https://mlsanity.com/paper/2508.00265/) (2025)
- [TSP3D: Text-guided Sparse Voxel Pruning for Efficient 3D Visual Grounding](https://mlsanity.com/paper/2502.10392/) (2025)
- [PanoGrounder: Bridging 2D and 3D with Panoramic Scene Representations for VLM-based 3D Visual Grounding](https://mlsanity.com/paper/2512.20907/) (2025)
- [Zero-Shot 3D Visual Grounding from Vision-Language Models](https://mlsanity.com/paper/2505.22429/) (2025)
- [FreeQ-Graph: Free-form Querying with Semantic Consistent Scene Graph for 3D Scene Understanding](https://mlsanity.com/paper/2506.13629/) (2025)
- [3D-DRES: Detailed 3D Referring Expression Segmentation](https://mlsanity.com/paper/2603.02896/) (2026)
- [UniGround: Universal 3D Visual Grounding via Training-Free Scene Parsing](https://mlsanity.com/paper/2603.08131/) (2026)
- [Think, Act, Build: An Agentic Framework with Vision Language Models for Zero-Shot 3D Visual Grounding](https://mlsanity.com/paper/2604.00528/) (2026)
- [3DWG: 3D Weakly Supervised Visual Grounding via Category and Instance-Level Alignment](https://mlsanity.com/paper/2505.01809/) (2025)
- [LSVG: Language-Guided Scene Graphs with 2D-Assisted Multi-Modal Encoding for 3D Visual Grounding](https://mlsanity.com/paper/2505.04058/) (2025)
- [B2N3D: Progressive Learning from Binary to N-ary Relationships for 3D Object Grounding](https://mlsanity.com/paper/2510.10194/) (2025)
- [OpenGround: Planning-based Online Perception for Open-World 3D Visual Grounding](https://mlsanity.com/paper/2512.23020/) (2025)
- [Reasoning Matters for 3D Visual Grounding](https://mlsanity.com/paper/2601.08811/) (2026)
- [Hierarchical Collaborative Fusion for 3D Instance-aware Referring Expression Segmentation](https://mlsanity.com/paper/2603.06250/) (2026)
- [PC-CrossDiff: Point-Cluster Dual-Level Cross-Modal Differential Attention for Unified 3D Referring and Segmentation](https://mlsanity.com/paper/2603.17753/) (2026)
- [Multiple Consistent 2D-3D Mappings for Robust Zero-Shot 3D Visual Grounding](https://mlsanity.com/paper/2604.26261/) (2026)
- [SceneGraphGrounder: Zero-Shot 3D Visual Grounding via Structured Scene Graph Matching](https://mlsanity.com/paper/2605.21788/) (2026)
- [AgentGrounder: Zero-Shot 3D Visual Pointcloud Grounding using Multimodal Language Models](https://mlsanity.com/paper/2605.25901/) (2026)
- [PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding](https://mlsanity.com/paper/2606.31148/) (2026)

## Abstract (arXiv)

3D referring expression comprehension (3DREC) and segmentation (3DRES) have overlapping objectives, indicating their potential for collaboration. However, existing collaborative approaches predominantly depend on the results of one task to make predictions for the other, limiting effective collaboration. We argue that employing separate branches for 3DREC and 3DRES tasks enhances the model's capacity to learn specific information for each task, enabling them to acquire complementary knowledge. Thus, we propose the MCLN framework, which includes independent branches for 3DREC and 3DRES tasks. This enables dedicated exploration of each task and effective coordination between the branches. Furthermore, to facilitate mutual reinforcement between these branches, we introduce a Relative Superpoint Aggregation (RSA) module and an Adaptive Soft Alignment (ASA) module. These modules significantly contribute to the precise alignment of prediction results from the two branches, directing the module to allocate increased attention to key positions. Comprehensive experimental evaluation demonstrates that our proposed method achieves state-of-the-art performance on both the 3DREC and 3DRES tasks, with an increase of 2.05% in Acc@0.5 for 3DREC and 3.96% in mIoU for 3DRES.