# OpenVox: Real-time Instance-level Open-vocabulary Probabilistic Voxel Representation

arXiv: https://arxiv.org/abs/2502.16528 · HTML: https://mlsanity.com/paper/2502.16528/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- First posted on arXiv 23 Feb 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Built on by (most cited first, 8 of 8)

- [OVI-MAP:Open-Vocabulary Instance-Semantic Mapping](https://mlsanity.com/paper/2603.26541/) (2026)
- [Think While You Map: Asynchronous Vision-Language Agents for Incremental 3D Scene Graphs](https://mlsanity.com/paper/2606.31471/) (2026)
- [FUS3DMaps: Scalable and Accurate Open-Vocabulary Semantic Mapping by 3D Fusion of Voxel- and Instance-Level Layers](https://mlsanity.com/paper/2605.03669/) (2026)
- [RoboAtlas: Contextual Active SLAM](https://mlsanity.com/paper/2606.26046/) (2026)
- [LatentAM: Real-Time, Large-Scale Latent Gaussian Attention Mapping via Online Dictionary Learning](https://mlsanity.com/paper/2602.12314/) (2026)
- [VOIM: Training-Free Open-Vocabulary 3D Instance Mapping for RGB-D and Monocular SLAM](https://mlsanity.com/paper/2609.00775/) (2026)
- [PerSeM: Persistent Semantic Memory for Long-Horizon Open-Vocabulary UAV Mapping](https://mlsanity.com/paper/2609.19542/) (2026)
- [TRACKGRAPH: Online Open-Vocabulary 3D Scene Graphs via Image-Space Tracking](https://mlsanity.com/paper/2609.31005/) (2026)

## Abstract (arXiv)

In recent years, vision-language models (VLMs) have advanced open-vocabulary mapping, enabling mobile robots to simultaneously achieve environmental reconstruction and high-level semantic understanding. While integrated object cognition helps mitigate semantic ambiguity in point-wise feature maps, efficiently obtaining rich semantic understanding and robust incremental reconstruction at the instance-level remains challenging. To address these challenges, we introduce OpenVox, a real-time incremental open-vocabulary probabilistic instance voxel representation. In the front-end, we design an efficient instance segmentation and comprehension pipeline that enhances language reasoning through encoding captions. In the back-end, we implement probabilistic instance voxels and formulate the cross-frame incremental fusion process into two subtasks: instance association and live map evolution, ensuring robustness to sensor and segmentation noise. Extensive evaluations across multiple datasets demonstrate that OpenVox achieves state-of-the-art performance in zero-shot instance segmentation, semantic segmentation, and open-vocabulary retrieval. Furthermore, real-world robotics experiments validate OpenVox's capability for stable, real-time operation.