mlsanityData to 7 Oct 2026

3D-GRES: Generalized 3D Referring Expression Segmentation

arXiv 2407.20664 ↗First posted 30 Jul 2024cs.CVcited by 18

Not on the map. Why?

Abstract (arXiv)

3D Referring Expression Segmentation (3D-RES) is dedicated to segmenting a specific instance within a 3D space based on a natural language description. However, current approaches are limited to segmenting a single target, restricting the versatility of the task. To overcome this limitation, we introduce Generalized 3D Referring Expression Segmentation (3D-GRES), which extends the capability to segment any number of instances based on natural language instructions. In addressing this broader task, we propose the Multi-Query Decoupled Interaction Network (MDIN), designed to break down multi-object segmentation tasks into simpler, individual segmentations. MDIN comprises two fundamental components: Text-driven Sparse Queries (TSQ) and Multi-object Decoupling Optimization (MDO). TSQ generates sparse point cloud features distributed over key targets as the initialization for queries. Meanwhile, MDO is tasked with assigning each target in multi-object scenarios to different queries while maintaining their semantic consistency. To adapt to this new task, we build a new dataset, namely Multi3DRes. Our comprehensive evaluations on this dataset demonstrate substantial enhancements over existing models, thus charting a new path for intricate multi-object 3D scene comprehension. The benchmark and code are available at https://github.com/sosppxo/MDIN.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 57 of its 79 references that are in the map. First posted on arXiv 30 Jul 2024.

Builds on

57 of 79 references.

References · 57

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020End-to-End Object Detection with Transformerscited by 7,384
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  5. 2016Modeling Context in Referring Expressionscited by 1,054
  6. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  7. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  8. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  9. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  10. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
  11. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 427
  12. 2017Large-scale Point Cloud Semantic Segmentation with Superpoint Graphscited by 384
38 more
  1. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 366
  2. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensingcited by 357
  3. 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 279
  4. 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
  5. 2016Segmentation from Natural Language Expressionscited by 257
  6. 2016Modeling Context Between Objects for Referring Expression Understandingcited by 250
  7. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 239
  8. 2021Group-Free 3D Object Detection via Transformerscited by 215
  9. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 142
  10. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 140
  11. 2018Learning to Assemble Neural Module Tree Networks for Visual Groundingcited by 139
  12. 2020Improving One-stage Visual Grounding by Recursive Sub-query Constructioncited by 137
  13. 2015Natural Language Object Retrievalcited by 124
  14. 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 121
  15. 2022Superpoint Transformer for 3D Scene Instance Segmentationcited by 115
  16. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 110
  17. 20223D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selectioncited by 109
  18. 2018Neighbourhood Watch: Referring Expression Comprehension via Language-guided Graph Attention Networkscited by 102
  19. 2024PSALM: Pixelwise SegmentAtion with Large Multi-Modal Modelcited by 100
  20. 2019Learning to Compose and Reason with Language Tree Structures for Visual Groundingcited by 98
  21. 2021Locate then Segment: A Strong Pipeline for Referring Image Segmentationcited by 98
  22. 2022EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Groundingcited by 98
  23. 2019Improving Referring Expression Grounding with Cross-modal Attention-guided Erasingcited by 88
  24. 2021TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Groundingcited by 86
  25. 2019Zero-Shot Grounding of Objects from Natural Language Queriescited by 79
  26. 2016A Joint Speaker-Listener-Reinforcer Model for Referring Expressionscited by 74
  27. 2021Back-tracing Representative Points for Voting-based 3D Object Detection in Point Cloudscited by 67
  28. 2024An Open and Comprehensive Pipeline for Unified Object Grounding and Detectioncited by 64
  29. 2017Parallel Attention: A Unified Framework for Visual Object Discovery through Dialogs and Queriescited by 56
  30. 2022Instance-Specific Feature Propagation for Referring Segmentationcited by 45
  31. 2023Mask-Attention-Free Transformer for 3D Instance Segmentationcited by 45
  32. 2023Described Object Detection: Liberating Object Detection with Flexible Expressionscited by 40
  33. 2023X-Mesh: Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidancecited by 37
  34. 20233D-STMN: Dependency-Driven Superpoint-Text Matching Network for End-to-End 3D Referring Expression Segmentationcited by 36
  35. 2023Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMscited by 35
  36. 2024Enhancing Video-Language Representations with Structural Spatio-Temporal Alignmentcited by 32
  37. 2023Scene Graph as Pivoting: Inference-time Image-free Unsupervised Multimodal Machine Translation with Visual Scene Hallucinationcited by 27
  38. 2023Towards Local Visual Modeling for Image Captioningcited by 26

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 9; 2026: 7.

18 papers.

Sort:

By area:

  1. 2025Multimodal Referring Segmentation: A Surveycited by 22
  2. 2024RG-SAN: Rule-Guided Spatial Awareness Network for End-to-End 3D Referring Expression Segmentationcited by 11
  3. 2025Hierarchical Alignment-enhanced Adaptive Grounding Network for Generalized Referring Expression Comprehensioncited by 11
  4. 2024Multimodal 3D Reasoning Segmentation with Complex Scenescited by 8
  5. 2026MVGGT: Multimodal Visual Geometry Grounded Transformer for Multiview 3D Referring Expression Segmentationcited by 8
  6. 2026LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generationcited by 6
  7. 2025PVUW 2025 Challenge Report: Advances in Pixel-level Understanding of Complex Videos in the Wildcited by 5
  8. 2025LSVOS 2025 Challenge Report: Recent Advances in Complex Video Object Segmentationcited by 4
  9. 20253DResT: A Strong Baseline for Semi-Supervised 3D Referring Expression Segmentationcited by 2
  10. 2025Point Linguist Model: Segment Any Object via Bridged Large 3D-Language Modelcited by 2
  11. 2025LIHE: Linguistic Instance-Split Hyperbolic-Euclidean Framework for Generalized Weakly-Supervised Referring Expression Comprehensioncited by 1
  12. 20263D-DRES: Detailed 3D Referring Expression Segmentationcited by 1
6 more
  1. 20263D Segmentation Using Viewpoint-Dependent Spatial Relationshipscited by 1
  2. 2025Proxy-Embedding as an Adversarial Teacher: An Embedding-Guided Bidirectional Attack for Referring Expression Segmentation Modelscited by 0
  3. 2025DenseScan: Advancing 3D Scene Understanding with 2D Dense Annotationcited by 0
  4. 2026Hierarchical Collaborative Fusion for 3D Instance-aware Referring Expression Segmentationcited by 0
  5. 2026Grounding by Remembering: Cross-Scene and In-Scene Memory for 3D Functional Affordancescited by 0
  6. 2026TrackRef3D: Multi-View Consistent Track-then-Label for Open-World Referring Segmentation in 3D Gaussian Splattingcited by 0