# Depth-Only Open-Vocabulary 3D Semantic Segmentation For Privacy-Preserving Robotic Applications

arXiv: https://arxiv.org/abs/2607.00978 · HTML: https://mlsanity.com/paper/2607.00978/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 41 references that are in the map.
- First posted on arXiv 1 Jul 2026.

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [OpenScene: 3D Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2211.15654/) (2022)
- [Scaling Open-Vocabulary Image Segmentation with Image-Level Labels](https://mlsanity.com/paper/2112.12143/) (2021)
- [ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning](https://mlsanity.com/paper/2309.16650/) (2023)
- [OpenMask3D: Open-Vocabulary 3D Instance Segmentation](https://mlsanity.com/paper/2306.13631/) (2023)
- [Language-Grounded Indoor 3D Semantic Segmentation in the Wild](https://mlsanity.com/paper/2204.07761/) (2022)
- [SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving](https://mlsanity.com/paper/2003.03653/) (2020)
- [PLA: Language-Driven Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2211.16312/) (2022)
- [Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance](https://mlsanity.com/paper/2312.10671/) (2023)
- [SAMPro3D: Locating SAM Prompts in 3D for Zero-Shot Instance Segmentation](https://mlsanity.com/paper/2311.17707/) (2023)
- [Search3D: Hierarchical Open-Vocabulary 3D Segmentation](https://mlsanity.com/paper/2409.18431/) (2024)
- [Open-world Semantic Segmentation for LIDAR Point Clouds](https://mlsanity.com/paper/2207.01452/) (2022)
- [Mosaic3D: Foundation Dataset and Model for Open-Vocabulary 3D Segmentation](https://mlsanity.com/paper/2502.02548/) (2025)
- [Hierarchical Point-based Active Learning for Semi-supervised Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2308.11166/) (2023)
- [Robust Depth-based Person Re-identification](https://mlsanity.com/paper/1703.09474/) (2017)
- [Open Vocabulary 3D Scene Understanding via Geometry Guided Self-Distillation](https://mlsanity.com/paper/2407.13362/) (2024)
- [LiDAL: Inter-frame Uncertainty Based Active Learning for 3D LiDAR Semantic Segmentation](https://mlsanity.com/paper/2211.05997/) (2022)
- [PP-OCRv6: From 1.5M to 34.5M Parameters, Surpassing Billion-Scale VLMs on OCR Tasks](https://mlsanity.com/paper/2606.13108/) (2026)
- [Cross-Modal and Uncertainty-Aware Agglomeration for Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2503.16707/) (2025)
- [Open-Vocabulary 3D Semantic Segmentation with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2407.13642/) (2024)
- [Masked Point-Entity Contrast for Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2504.19500/) (2025)
- [Open-Vocabulary SAM3D: Towards Training-free Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2405.15580/) (2024)
- [SAS: Segment Any 3D Scene with Integrated 2D Priors](https://mlsanity.com/paper/2503.08512/) (2025)
- [XMask3D: Cross-modal Mask Reasoning for Open Vocabulary 3D Semantic Segmentation](https://mlsanity.com/paper/2411.13243/) (2024)
- [MOSAIC: Generating Consistent, Privacy-Preserving Scenes from Multiple Depth Views in Multi-Room Environments](https://mlsanity.com/paper/2503.13816/) (2025)
- [LOSC: LiDAR Open-voc Segmentation Consolidator](https://mlsanity.com/paper/2507.07605/) (2025)
- [Designing Privacy-Preserving Visual Perception for Robot Navigation Based on User Privacy Preferences](https://mlsanity.com/paper/2604.06382/) (2026)

## Built on by (most cited first, 1 of 1)

- [Privacy-Preserving Semantic Segmentation from High-Resolution Depth and Ultra-Low-Resolution RGB](https://mlsanity.com/paper/2609.28360/) (2026)

## Abstract (arXiv)

Privacy-preserving perception is increasingly important for robotic systems operating in real-world indoor environments, yet it remains underexplored in open-vocabulary 3D semantic segmentation. We study this problem under an RGB-prohibited deployment setting motivated by scene-specific visual information disclosure, where real RGB observations are unavailable during scene acquisition and fusion. To reflect this deployment constraint on existing 3D datasets, we adopt a stricter depth-only evaluation protocol that re-runs scene fusion without RGB and exposes only the resulting depth-derived geometry to the segmentation pipeline. This constraint removes appearance cues that are often critical for open-vocabulary recognition, making depth-only predictions more uncertain and less reliable. To address this challenge, we propose UTTO, a model-agnostic uncertainty-guided test-time optimization framework that uses structured predictive uncertainty as a reliability signal to refine predictions from frozen open-vocabulary 3D backbones. Experiments across ScanNet and Matterport3D demonstrate consistent improvements over multiple depth-only backbones. Privacy recoverability analyses and a real-robot semantic goal grounding case study further support the proposed privacy-constrained setting and applicability.