mlsanityData to 7 Oct 2026

KM-ViPE: Online Tightly Coupled Vision-Language-Geometry Fusion for Open-Vocabulary Semantic SLAM

arXiv 2512.01889 ↗First posted 1 Dec 2025cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

We present KM-ViPE (Knowledge Mapping Video Pose Engine), a real-time open-vocabulary SLAM framework for uncalibrated monocular cameras in dynamic environments. Unlike systems requiring depth sensors and offline calibration, KM-ViPE operates directly on raw RGB streams, making it ideal for ego-centric applications and harvesting internet-scale video data for training. KM-ViPE tightly couples DINO visual features with geometric constraints through a high-level features based adaptive robust kernel that handles both moving objects and movable static objects (e.g., moving furniture in ego-centric views). The system performs simultaneous online localization and open-vocabulary semantic mapping by fusing geometric and deep visual features aligned with language embeddings. Our results are competitive with state-of-the-art approaches, while existing solutions either operate offline, need depth data and/or odometry estimation, or lack dynamic scene robustness. KM-ViPE benefits from internet-scale training and uniquely combines online operation, uncalibrated monocular input, and robust handling of dynamic scenes, which makes it a good fit for autonomous robotics and AR/VR applications and advances practical spatial intelligence capabilities for embodied AI.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 38 of its 44 references that are in the map. First posted on arXiv 1 Dec 2025.

Builds on

38 of 44 references.

References · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Segment Anythingcited by 8,158
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  5. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  6. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  7. 2025DINOv3cited by 1,297
  8. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  9. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 862
  10. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
  11. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
  12. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
26 more
  1. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  2. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
  3. 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
  4. 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 288
  5. 2017A General and Adaptive Robust Loss Functioncited by 236
  6. 2024MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priorscited by 198
  7. 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 195
  8. 2019Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mappingcited by 181
  9. 2023Aria Digital Twin: A New Benchmark Dataset for Egocentric 3D Machine Perceptioncited by 166
  10. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  11. 2025VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifoldcited by 130
  12. 2023Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidancecited by 123
  13. 2024Clio: Real-time Task-Driven Open-Set 3D Scene Graphscited by 81
  14. 2024OpenNeRF: Open Set 3D Neural Scene Segmentation with Pixel-Wise Features and Rendered Novel Viewscited by 68
  15. 2024GeoCalib: Learning Single-image Calibration with Geometric Optimizationcited by 50
  16. 2020DynaSLAM II: Tightly-Coupled Multi-Object Tracking and SLAMcited by 47
  17. 2024Talking to DINO: Bridging Self-Supervised Vision Backbones with Language for Open-Vocabulary Segmentationcited by 42
  18. 2024Beyond Bare Queries: Open-Vocabulary Object Grounding with 3D Scene Graphcited by 32
  19. 2024AirSLAM: An Efficient and Illumination-Robust Point-Line Visual SLAM Systemcited by 26
  20. 2024RoDyn-SLAM: Robust Dynamic Dense RGB-D SLAM with Neural Radiance Fieldscited by 22
  21. 2020Adaptive Robust Kernels for Non-Linear Least Squares Problemscited by 13
  22. 2024RGBDS-SLAM: A RGB-D Semantic Dense SLAM Based on 3D Multi Level Pyramid Gaussian Splattingcited by 6
  23. 2022Probabilistic Data Association for Semantic SLAM at Scalecited by 4
  24. 2024V3D-SLAM: Robust RGB-D SLAM in Dynamic Environments with 3D Semantic Geometry Votingcited by 3
  25. 2025OSMa-Bench: Evaluating Open Semantic Mapping Under Varying Lighting Conditionscited by 2
  26. 2025FlowFeat: Pixel-Dense Embedding of Motion Profilescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026VOIM: Training-Free Open-Vocabulary 3D Instance Mapping for RGB-D and Monocular SLAMcited by 0