mlsanityData to 7 Oct 2026

Ges3ViG: Incorporating Pointing Gestures into Language-Based 3D Visual Grounding for Embodied Reference Understanding

arXiv 2504.09623 ↗First posted 13 Apr 2025cs.CV cs.AI cs.MMcited by 7

Not on the map. Why?

Abstract (arXiv)

3-Dimensional Embodied Reference Understanding (3D-ERU) combines a language description and an accompanying pointing gesture to identify the most relevant target object in a 3D scene. Although prior work has explored pure language-based 3D grounding, there has been limited exploration of 3D-ERU, which also incorporates human pointing gestures. To address this gap, we introduce a data augmentation framework-Imputer, and use it to curate a new benchmark dataset-ImputeRefer for 3D-ERU, by incorporating human pointing gestures into existing 3D scene datasets that only contain language instructions. We also propose Ges3ViG, a novel model for 3D-ERU that achieves ~30% improvement in accuracy as compared to other 3D-ERU models and ~9% compared to other purely language-based 3D grounding models. Our code and dataset are available at https://github.com/AtharvMane/Ges3ViG.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 24 of its 47 references that are in the map. First posted on arXiv 13 Apr 2025.

Builds on

24 of 47 references.

References · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  5. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  6. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
  7. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 427
  8. 2020PointGroup: Dual-Set Point Grouping for 3D Instance Segmentationcited by 318
  9. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 269
  10. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 142
  11. 2022Multi-View Transformer for 3D Visual Groundingcited by 138
  12. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
12 more
  1. 2015Grounding of Textual Phrases in Images by Reconstructioncited by 112
  2. 2021Free-form Description Guided 3D Visual Graph Network for Object Grounding in Point Cloudcited by 81
  3. 2022UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Groundingcited by 69
  4. 2023NS3D: Neuro-Symbolic Grounding of 3D Objects and Relationscited by 57
  5. 2021D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Groundingcited by 52
  6. 2022Look Around and Refer: 2D Synthetic Semantics Knowledge Distillation for 3D Visual Groundingcited by 41
  7. 2020Point2Skeleton: Learning Skeletal Representations from Point Cloudscited by 37
  8. 2021YouRefIt: Embodied Reference Understanding with Language and Gesturecited by 32
  9. 2023SMPLitex: A Generative Model and Dataset for 3D Human Texture Estimation from Single Imagecited by 30
  10. 2024RefMask3D: Language-Guided Transformer for 3D Referring Segmentationcited by 25
  11. 2023ScanERU: Interactive 3D Visual Grounding based on Embodied Reference Understandingcited by 12
  12. 2022Learning Point-Language Hierarchical Alignment for 3D Visual Groundingcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2026Egocentric Co-Pilot: Web-Native Smart-Glasses Agents for Assistive Egocentric AIcited by 3
  2. 2025CAPE: A CLIP-Aware Pointing Ensemble of Complementary Heatmap Cues for Embodied Reference Understandingcited by 0
  3. 2025A Multimodal Depth-Aware Method For Embodied Reference Understandingcited by 0
  4. 2026PoseRefer: Pathway-Local Parameters for Semantically Grounded Reference Resolutioncited by 0
  5. 2026PruneGround: Plug-and-play Spatial Pruning for 3D Visual Groundingcited by 0
  6. 2026Talk2Sensors: 3D Visual Grounding in Autonomous Driving via Sensor-Adaptive Physical Cue Matchingcited by 0
  7. 2026CL4D: Contrastive Language-4D Pretraining for Vision-Language Reasoning in Dynamic Scenescited by 0