mlsanityData to 14 Oct 2026

Ov3R: Open-Vocabulary Semantic 3D Reconstruction from RGB Videos

arXiv 2507.22052 ↗First posted 29 Jul 2025cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

We present Ov3R, a novel framework for open-vocabulary semantic 3D reconstruction from RGB video streams, designed to advance Spatial AI. The system features two key components: CLIP3R, a CLIP-informed 3D reconstruction module that predicts dense point maps from overlapping clips while embedding object-level semantics; and 2D-3D OVS, a 2D-3D open-vocabulary semantic module that lifts 2D features into 3D by learning fused descriptors integrating spatial, geometric, and semantic cues. Unlike prior methods, Ov3R incorporates CLIP semantics directly into the reconstruction process, enabling globally consistent geometry and fine-grained semantic alignment. Our framework achieves state-of-the-art performance in both dense 3D reconstruction and open-vocabulary 3D segmentation, marking a step forward toward real-time, semantics-aware Spatial AI.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 53 of its 69 references that are in the map. First posted on arXiv 29 Jul 2025.

Builds on

53 of 69 references.

References · 53

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023Segment Anythingcited by 8,234
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  7. 2021Vision Transformers for Dense Predictioncited by 1,803
  8. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  9. 2024Grounding Image Matching in 3D with MASt3Rcited by 983
  10. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  11. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 878
  12. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
38 more
  1. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
  2. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  3. 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 616
  4. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 566
  5. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  6. 2023LERF: Language Embedded Radiance Fieldscited by 543
  7. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 473
  8. 2023Neuralangelo: High-Fidelity Neural Surface Reconstructioncited by 467
  9. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
  10. 2023LangSplat: 3D Language Gaussian Splattingcited by 415
  11. 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 299
  12. 2023Fast Segment Anythingcited by 290
  13. 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 289
  14. 2024PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstructioncited by 285
  15. 20243D Reconstruction with Spatial Memorycited by 277
  16. 2021Pixel-Perfect Structure-from-Motion with Featuremetric Refinementcited by 200
  17. 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 196
  18. 2023NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAMcited by 164
  19. 2016SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networkscited by 135
  20. 2023GO-SLAM: Global Optimization for Consistent 3D Instant Reconstructioncited by 133
  21. 2023Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidancecited by 124
  22. 2024MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Viewscited by 116
  23. 2024SLAM3R: Real-Time Dense Scene Reconstruction from Monocular RGB Videoscited by 102
  24. 2024SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Modelcited by 96
  25. 2024SGS-SLAM: Semantic Gaussian Splatting For Neural Dense SLAMcited by 87
  26. 2023CLIP goes 3D: Leveraging Prompt Tuning for Language Grounded 3D Recognitioncited by 77
  27. 2024How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Surveycited by 72
  28. 2024OpenNeRF: Open Set 3D Neural Scene Segmentation with Pixel-Wise Features and Rendered Novel Viewscited by 69
  29. 2023SNI-SLAM: Semantic Neural Implicit SLAMcited by 65
  30. 2023Dense RGB SLAM with Neural Implicit Mapscited by 42
  31. 2024SemGauss-SLAM: Dense Semantic Gaussian Splatting SLAMcited by 42
  32. 20233D Line Mapping Revisitedcited by 39
  33. 2018FutureMapping: The Computational Structure of Spatial AI Systemscited by 36
  34. 2024NIS-SLAM: Neural Implicit Semantic RGB-D SLAM for 3D Consistent Scene Understandingcited by 19
  35. 2024Learning-based Multi-View Stereo: A Surveycited by 18
  36. 2024NEDS-SLAM: A Neural Explicit Dense Semantic SLAM Framework using 3D Gaussian Splattingcited by 16
  37. 2024MGS-SLAM: Monocular Sparse Tracking and Gaussian Mapping with Depth Smooth Regularizationcited by 10
  38. 2024O2V-Mapping: Online Open-Vocabulary Mapping with Neural Implicit Representationcited by 9

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 8.

10 papers.

  1. 2025GaussianDWM: 3D Gaussian Driving World Model for Unified Scene Understanding and Multi-Modal Generationcited by 23
  2. 2025Unified Semantic Transformer for 3D Scene Understandingcited by 8
  3. 2026Pano3D: Unified 3D Reconstruction and Panoptic Segmentationcited by 1
  4. 2026Genie 4D: Semantic-Prior-Guided 4D Dynamic Scene Reconstructioncited by 0
  5. 2026Cross-Attentive Multiview Fusion of Vision-Language Embeddingscited by 0
  6. 2026Bidirectional Cross-Modal Prompting for Event-Frame Asymmetric Stereocited by 0
  7. 2026RSGPNet: Geometric Prompting for Remote Sensing Open-Vocabulary Semantic Segmentationcited by 0
  8. 2026MAGiSt3R: Multi-Agent Feed-forward 3D Reconstruction from Monocular RGB Videoscited by 0
  9. 2026GaussianDWM++: Language-Grounded 3D Gaussian Driving World Model for Unified Scene Understanding, Editing, and Multi-Modal Generationcited by 0
  10. 2026VOIM: Training-Free Open-Vocabulary 3D Instance Mapping for RGB-D and Monocular SLAMcited by 0