# AugRefer: Advancing 3D Visual Grounding via Cross-Modal Augmentation and Spatial Relation-based Referring

arXiv: https://arxiv.org/abs/2501.09428 · HTML: https://mlsanity.com/paper/2501.09428/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 16 Jan 2025.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language](https://mlsanity.com/paper/1912.08830/) (2019)
- [Group-Free 3D Object Detection via Transformers](https://mlsanity.com/paper/2104.00678/) (2021)
- [PLA: Language-Driven Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2211.16312/) (2022)
- [Multi3DRefer: Grounding Text Description to Multiple 3D Objects](https://mlsanity.com/paper/2309.05251/) (2023)
- [InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring](https://mlsanity.com/paper/2103.01128/) (2021)
- [Multi-View Transformer for 3D Visual Grounding](https://mlsanity.com/paper/2204.02174/) (2022)
- [Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Clouds](https://mlsanity.com/paper/2112.08879/) (2021)
- [Mix3D: Out-of-Context Data Augmentation for 3D Scenes](https://mlsanity.com/paper/2110.02210/) (2021)
- [SAT: 2D Semantics Assisted Training for 3D Visual Grounding](https://mlsanity.com/paper/2105.11450/) (2021)
- [Language Conditioned Spatial Relation Reasoning for 3D Object Grounding](https://mlsanity.com/paper/2211.09646/) (2022)
- [3D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selection](https://mlsanity.com/paper/2204.06272/) (2022)
- [EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Grounding](https://mlsanity.com/paper/2209.14941/) (2022)
- [SESS: Self-Ensembling Semi-Supervised 3D Object Detection](https://mlsanity.com/paper/1912.11803/) (2019)
- [ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding with GPT and Prototype Guidance](https://mlsanity.com/paper/2303.16894/) (2023)
- [Distilling Coarse-to-Fine Semantic Matching Knowledge for Weakly Supervised 3D Visual Grounding](https://mlsanity.com/paper/2307.09267/) (2023)
- [3D Copy-Paste: Physically Plausible Object Insertion for Monocular 3D Detection](https://mlsanity.com/paper/2312.05277/) (2023)
- [DODA: Data-oriented Sim-to-Real Domain Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2204.01599/) (2022)
- [TrAME: Trajectory-Anchored Multi-View Editing for Text-Guided 3D Gaussian Splatting Manipulation](https://mlsanity.com/paper/2407.02034/) (2024)
- [Synthetic-to-Real Domain Generalized Semantic Segmentation for 3D Indoor Point Clouds](https://mlsanity.com/paper/2212.04668/) (2022)

## Built on by (most cited first, 10 of 10)

- [Robust Multi-View Learning via Representation Fusion of Sample-Level Attention and Alignment of Simulated Perturbation](https://mlsanity.com/paper/2503.04151/) (2025)
- [RaLiFlow: Scene Flow Estimation with 4D Radar and LiDAR Point Clouds](https://mlsanity.com/paper/2512.10376/) (2025)
- [Graph Smoothing for Enhanced Local Geometry Learning in Point Cloud Analysis](https://mlsanity.com/paper/2601.11102/) (2026)
- [3DResT: A Strong Baseline for Semi-Supervised 3D Referring Expression Segmentation](https://mlsanity.com/paper/2504.12599/) (2025)
- [Few-Shot Incremental 3D Object Detection in Dynamic Indoor Environments](https://mlsanity.com/paper/2604.07997/) (2026)
- [Scalable Object Relation Encoding for Better 3D Spatial Reasoning in Large Language Models](https://mlsanity.com/paper/2603.24721/) (2026)
- [PC-CrossDiff: Point-Cluster Dual-Level Cross-Modal Differential Attention for Unified 3D Referring and Segmentation](https://mlsanity.com/paper/2603.17753/) (2026)
- [Multiple Consistent 2D-3D Mappings for Robust Zero-Shot 3D Visual Grounding](https://mlsanity.com/paper/2604.26261/) (2026)
- [CAPruner: Conceptual-Adjacent Scene Graph Pruner for Enhancing 3D Spatial Reasoning of Large Language Models](https://mlsanity.com/paper/2606.07529/) (2026)
- [PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding](https://mlsanity.com/paper/2606.31148/) (2026)

## Abstract (arXiv)

3D visual grounding (3DVG), which aims to correlate a natural language description with the target object within a 3D scene, is a significant yet challenging task. Despite recent advancements in this domain, existing approaches commonly encounter a shortage: a limited amount and diversity of text3D pairs available for training. Moreover, they fall short in effectively leveraging different contextual clues (e.g., rich spatial relations within the 3D visual space) for grounding. To address these limitations, we propose AugRefer, a novel approach for advancing 3D visual grounding. AugRefer introduces cross-modal augmentation designed to extensively generate diverse text-3D pairs by placing objects into 3D scenes and creating accurate and semantically rich descriptions using foundation models. Notably, the resulting pairs can be utilized by any existing 3DVG methods for enriching their training data. Additionally, AugRefer presents a language-spatial adaptive decoder that effectively adapts the potential referring objects based on the language description and various 3D spatial relations. Extensive experiments on three benchmark datasets clearly validate the effectiveness of AugRefer.