mlsanityData to 7 Oct 2026

Mono3DVG: 3D Visual Grounding in Monocular Images

arXiv 2312.08022 ↗First posted 13 Dec 2023cs.CVcited by 22

Not on the map. Why?

Abstract (arXiv)

We introduce a novel task of 3D visual grounding in monocular RGB images using language descriptions with both appearance and geometry information. Specifically, we build a large-scale dataset, Mono3DRefer, which contains 3D object targets with their corresponding geometric text descriptions, generated by ChatGPT and refined manually. To foster this task, we propose Mono3DVG-TR, an end-to-end transformer-based network, which takes advantage of both the appearance and geometry information in text embeddings for multi-modal learning and 3D object localization. Depth predictor is designed to explicitly learn geometry features. The dual text-guided adapter is proposed to refine multiscale visual and geometry features of the referred object. Based on depth-text-visual stacking attention, the decoder fuses object-level geometric cues and visual appearance into a learnable query. Comprehensive benchmarks and some insightful analyses are provided for Mono3DVG. Extensive comparisons and ablation studies show that our method significantly outperforms all baselines. The dataset and code will be publicly available at: https://github.com/ZhanYang-nwpu/Mono3DVG.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 45 of its 53 references that are in the map. First posted on arXiv 13 Dec 2023.

Builds on

45 of 53 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  4. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
  5. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
  6. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  7. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 427
  8. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  9. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 269
  10. 2019M3D-RPN: Monocular 3D Region Proposal Network for Object Detectioncited by 251
  11. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 248
  12. 2020SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimationcited by 202
33 more
  1. 2019A Fast and Accurate One-Stage Approach to Visual Groundingcited by 194
  2. 2021Objects are Different: Flexible Monocular 3D Object Detectioncited by 184
  3. 2019Learning Depth-Guided Convolutions for Monocular 3D Object Detectioncited by 169
  4. 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 159
  5. 2020MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationshipscited by 153
  6. 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
  7. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 140
  8. 2018Learning to Assemble Neural Module Tree Networks for Visual Groundingcited by 139
  9. 2021Delving into Localization Errors for Monocular 3D Object Detectioncited by 138
  10. 2022Multi-View Transformer for 3D Visual Groundingcited by 138
  11. 2020Improving One-stage Visual Grounding by Recursive Sub-query Constructioncited by 137
  12. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
  13. 2020Kinematic 3D Object Detection in Monocular Videocited by 116
  14. 2019Dynamic Graph Attention for Referring Expression Comprehensioncited by 111
  15. 2022MonoDTR: Monocular 3D Object Detection with Depth-Aware Transformercited by 108
  16. 2016Modeling Relationships in Referential Expressions with Compositional Modular Networkscited by 106
  17. 2018Neighbourhood Watch: Referring Expression Comprehension via Language-guided Graph Attention Networkscited by 102
  18. 2019Learning to Compose and Reason with Language Tree Structures for Visual Groundingcited by 98
  19. 2019Improving Referring Expression Grounding with Cross-modal Attention-guided Erasingcited by 88
  20. 2021TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Groundingcited by 86
  21. 2018MonoGRNet: A Geometric Reasoning Network for Monocular 3D Object Localizationcited by 84
  22. 2017Grounding Referring Expressions in Images by Variational Contextcited by 82
  23. 2021Free-form Description Guided 3D Visual Graph Network for Object Grounding in Point Cloudcited by 81
  24. 2019Zero-Shot Grounding of Objects from Natural Language Queriescited by 79
  25. 2022Improving Visual Grounding with Visual-Linguistic Verification and Iterative Reasoningcited by 69
  26. 2018Real-Time Referring Expression Comprehension by Single-Stage Grounding Networkcited by 56
  27. 2021D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Groundingcited by 52
  28. 2022Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Groundingcited by 52
  29. 2021Look Before You Leap: Learning Landmark Features for One-Stage Visual Groundingcited by 51
  30. 2018Rethinking Diversified and Discriminative Proposal Generation for Visual Groundingcited by 49
  31. 2017Query-guided Regression Network with Context Policy for Phrase Groundingcited by 34
  32. 2021Refer-it-in-RGBD: A Bottom-up Approach for 3D Visual Grounding in RGBD Imagescited by 33
  33. 2019Relationship-Embedded Representation Learning for Grounding Referring Expressionscited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 12; 2026: 4.

22 papers.

Sort:

By area:

  1. 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Modelcited by 112
  2. 2024A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directionscited by 24
  3. 2024Talk2Radar: Bridging Natural Language with 4D mmWave Radar for 3D Referring Expression Comprehensioncited by 17
  4. 2025NuGrounding: A Multi-View 3D Visual Grounding Framework in Autonomous Drivingcited by 12
  5. 20253EED: Ground Everything Everywhere in 3Dcited by 8
  6. 2024LLMI3D: MLLM-based 3D Perception from a Single 2D Imagecited by 7
  7. 2025Talk2Event: Grounded Understanding of Dynamic Scenes from Event Camerascited by 6
  8. 2024Talk to Parallel LiDARs: A Human-LiDAR Interaction Method Based on 3D Visual Groundingcited by 5
  9. 2025PanoGrounder: Bridging 2D and 3D with Panoramic Scene Representations for VLM-based 3D Visual Groundingcited by 3
  10. 2025Think Before You Drive: World Model-Inspired Multimodal Grounding for Autonomous Vehiclescited by 2
  11. 2024LidaRefer: Context-aware Outdoor 3D Visual Grounding for Autonomous Drivingcited by 1
  12. 2025Object Affordance Recognition and Grounding via Multi-scale Cross-modal Representation Learningcited by 1
10 more
  1. 2025A Coarse-to-Fine Approach to Multi-Modality 3D Occupancy Groundingcited by 1
  2. 2025Dual Enhancement on 3D Vision-Language Perception for Monocular 3D Visual Groundingcited by 1
  3. 2025Visual Grounding from Event Camerascited by 1
  4. 2025Talk2PC: Enhancing 3D Visual Grounding through LiDAR and Radar Point Clouds Fusion for Autonomous Drivingcited by 0
  5. 2025SpikeSMOKE: Spiking Neural Networks for Monocular 3D Object Detection with Cross-Scale Gated Codingcited by 0
  6. 2025B2N3D: Progressive Learning from Binary to N-ary Relationships for 3D Object Groundingcited by 0
  7. 2026Learning Human Visual Attention on 3D Surfaces through Geometry-Queried Semantic Priorscited by 0
  8. 2026Scaling Diverse Language Generation for 3D Visual Groundingcited by 0
  9. 2026PruneGround: Plug-and-play Spatial Pruning for 3D Visual Groundingcited by 0
  10. 2026Talk2Sensors: 3D Visual Grounding in Autonomous Driving via Sensor-Adaptive Physical Cue Matchingcited by 0