# Ref-SAM3D: Bridging SAM3D with Text for Reference 3D Reconstruction

arXiv: https://arxiv.org/abs/2511.19426 · HTML: https://mlsanity.com/paper/2511.19426/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 15 references that are in the map.
- First posted on arXiv 24 Nov 2025.

## Builds on (most cited first, 13 of 13)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation](https://mlsanity.com/paper/2212.04493/) (2022)
- [Anything-3D: Towards Single-view Anything Reconstruction in the Wild](https://mlsanity.com/paper/2304.10261/) (2023)
- [The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shot](https://mlsanity.com/paper/2306.16623/) (2023)
- [MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2508.05630/) (2025)
- [Part123: Part-aware 3D Reconstruction from a Single-view Image](https://mlsanity.com/paper/2405.16888/) (2024)
- [ShapeGPT: 3D Shape Generation with A Unified Multi-modal Language Model](https://mlsanity.com/paper/2311.17618/) (2023)
- [Prompting Segmentation with Sound Is Generalizable Audio-Visual Source Localizer](https://mlsanity.com/paper/2309.07929/) (2023)
- [Multimodal Referring Segmentation: A Survey](https://mlsanity.com/paper/2508.00265/) (2025)
- [LAM3D: Large Image-Point-Cloud Alignment Model for 3D Reconstruction from Single Image](https://mlsanity.com/paper/2405.15622/) (2024)
- [MTFusion: Reconstructing Any 3D Object from Single Image Using Multi-word Textual Inversion](https://mlsanity.com/paper/2411.12197/) (2024)

## Built on by (most cited first, 1 of 1)

- [LISA-3D: Lifting Language-Image Segmentation to 3D via Multi-View Consistency](https://mlsanity.com/paper/2512.01008/) (2025)

## Abstract (arXiv)

SAM3D has garnered widespread attention for its strong 3D object reconstruction capabilities. However, a key limitation remains: SAM3D cannot reconstruct specific objects referred to by textual descriptions, a capability that is essential for practical applications such as 3D editing, game development, and virtual environments. To address this gap, we introduce Ref-SAM3D, a simple yet effective extension to SAM3D that incorporates textual descriptions as a high-level prior, enabling text-guided 3D reconstruction from a single RGB image. Through extensive qualitative experiments, we show that Ref-SAM3D, guided only by natural language and a single 2D view, delivers competitive and high-fidelity zero-shot reconstruction performance. Our results demonstrate that Ref-SAM3D effectively bridges the gap between 2D visual cues and 3D geometric understanding, offering a more flexible and accessible paradigm for reference-guided 3D reconstruction. Code is available at: https://github.com/FudanCVL/Ref-SAM3D.