mlsanityData to 7 Oct 2026

RG-SAN: Rule-Guided Spatial Awareness Network for End-to-End 3D Referring Expression Segmentation

arXiv 2412.02402 ↗First posted 3 Dec 2024cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

3D Referring Expression Segmentation (3D-RES) aims to segment 3D objects by correlating referring expressions with point clouds. However, traditional approaches frequently encounter issues like over-segmentation or mis-segmentation, due to insufficient emphasis on spatial information of instances. In this paper, we introduce a Rule-Guided Spatial Awareness Network (RG-SAN) by utilizing solely the spatial information of the target instance for supervision. This approach enables the network to accurately depict the spatial relationships among all entities described in the text, thus enhancing the reasoning capabilities. The RG-SAN consists of the Text-driven Localization Module (TLM) and the Rule-guided Weak Supervision (RWS) strategy. The TLM initially locates all mentioned instances and iteratively refines their positional information. The RWS strategy, acknowledging that only target objects have supervised positional information, employs dependency tree rules to precisely guide the core instance's positioning. Extensive testing on the ScanRefer benchmark has shown that RG-SAN not only establishes new performance benchmarks, with an mIoU increase of 5.1 points, but also exhibits significant improvements in robustness when processing descriptions with spatial ambiguity. All codes are available at https://github.com/sosppxo/RG-SAN.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 64 of its 78 references that are in the map. First posted on arXiv 3 Dec 2024.

Builds on

64 of 78 references.

References · 64

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  7. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  9. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  10. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  11. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  12. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
38 more
  1. 2017Localizing Moments in Video with Natural Languagecited by 660
  2. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  3. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  4. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
  5. 2017Large-scale Point Cloud Semantic Segmentation with Superpoint Graphscited by 384
  6. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  7. 2021Rethinking and Improving Relative Position Encoding for Vision Transformercited by 206
  8. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 201
  9. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 170
  10. 2022Stratified Transformer for 3D Point Cloud Segmentationcited by 166
  11. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognitioncited by 151
  12. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 140
  13. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 135
  14. 2021Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Cloudscited by 128
  15. 2023Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenescited by 125
  16. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
  17. 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 121
  18. 2022ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehensioncited by 119
  19. 2022Superpoint Transformer for 3D Scene Instance Segmentationcited by 115
  20. 20223D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selectioncited by 109
  21. 2021Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networkscited by 98
  22. 2022EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Groundingcited by 98
  23. 2019Weakly Supervised Video Moment Retrieval From Text Queriescited by 85
  24. 2021Free-form Description Guided 3D Visual Graph Network for Object Grounding in Point Cloudcited by 81
  25. 2022Fine-Grained Semantically Aligned Vision-Language Pre-Trainingcited by 68
  26. 2021Person Re-identification Method Based on Color Attack and Joint Defencecited by 56
  27. 2024Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editingcited by 49
  28. 2020VQA-LOL: Visual Question Answering under the Lens of Logiccited by 48
  29. 2023Mask-Attention-Free Transformer for 3D Instance Segmentationcited by 45
  30. 2024Towards Semantic Equivalence of Tokenization in Multimodal LLMcited by 45
  31. 20233D-STMN: Dependency-Driven Superpoint-Text Matching Network for End-to-End 3D Referring Expression Segmentationcited by 36
  32. 2023Referring Image Segmentation Using Text Supervisioncited by 35
  33. 2024SegPoint: Segment Any Point Cloud via Large Language Modelcited by 34
  34. 2024Enhancing Video-Language Representations with Structural Spatio-Temporal Alignmentcited by 32
  35. 2024Reason3D: Searching and Reasoning 3D Segmentation via Large Language Modelcited by 27
  36. 2020WeaQA: Weak Supervision via Captions for Visual Question Answeringcited by 25
  37. 2022TOIST: Task Oriented Instance Segmentation Transformer with Noun-Pronoun Distillationcited by 25
  38. 2021Structured Multi-modal Feature Embedding and Alignment for Image-Sentence Retrievalcited by 23

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 5.

11 papers.

  1. 2025Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenescited by 62
  2. 2025Multimodal Referring Segmentation: A Surveycited by 22
  3. 2026MVGGT: Multimodal Visual Geometry Grounded Transformer for Multiview 3D Referring Expression Segmentationcited by 8
  4. 2025Language-to-Space Programming for Training-Free 3D Visual Groundingcited by 7
  5. 2025DEGround: An Effective Baseline for Ego-centric 3D Visual Grounding with a Homogeneous Frameworkcited by 4
  6. 20253DResT: A Strong Baseline for Semi-Supervised 3D Referring Expression Segmentationcited by 2
  7. 2025What is the Right Embedding Space for Contrastive Learning in Referring Expression Counting?cited by 2
  8. 2026HiFi-Mesh: High-Fidelity Efficient 3D Mesh Generation via Compact Autoregressive Dependencecited by 2
  9. 2026ZeroSplat: Generalized Referring Segmentation in 3D Gaussian Splattingcited by 2
  10. 2026Hierarchical Collaborative Fusion for 3D Instance-aware Referring Expression Segmentationcited by 0
  11. 2026TrackRef3D: Multi-View Consistent Track-then-Label for Open-World Referring Segmentation in 3D Gaussian Splattingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ColLab: A Collaborative Spatial Progressive Data Engine for Referring Expression Comprehension and Generation
  2. 2026SSR3D-LLM: Structured Spatial Reasoning via Latent Steps for Fine-Grained Grounding in Unified 3D-LLMs
  3. 2026ResAgent: Entropy-based Prior Point Discovery and Visual Reasoning for Referring Expression Segmentation
  4. 2026DRAgent: Discriminative Reasoning Agent for Referring Expression Segmentation
  5. 2025IPDN: Image-enhanced Prompt Decoding Network for 3D Referring Expression Segmentation
  6. 20243D Spatial Understanding in MLLMs: Disambiguation and Evaluation
  7. 2025Geo-R1: Improving Few-Shot Geospatial Referring Expression Understanding with Reinforcement Fine-Tuning
  8. 2026ChartREG++: Towards Benchmarking and Improving Chart Referring Expression Grounding under Diverse referring clues and Multi-Target Referring