# LOSC: LiDAR Open-voc Segmentation Consolidator

arXiv: https://arxiv.org/abs/2507.07605 · HTML: https://mlsanity.com/paper/2507.07605/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 33 of its 48 references that are in the map.
- First posted on arXiv 10 Jul 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 30 of 33)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [RegionCLIP: Region-based Language-Image Pretraining](https://mlsanity.com/paper/2112.09106/) (2021)
- [Point Transformer V3: Simpler, Faster, Stronger](https://mlsanity.com/paper/2312.10035/) (2023)
- [Extract Free Dense Labels from CLIP](https://mlsanity.com/paper/2112.01071/) (2021)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [OpenScene: 3D Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2211.15654/) (2022)
- [Segment Everything Everywhere All at Once](https://mlsanity.com/paper/2304.06718/) (2023)
- [Scaling Open-Vocabulary Image Segmentation with Image-Level Labels](https://mlsanity.com/paper/2112.12143/) (2021)
- [A Simple Framework for Open-Vocabulary Segmentation and Detection](https://mlsanity.com/paper/2303.08131/) (2023)
- [CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP](https://mlsanity.com/paper/2301.04926/) (2023)
- [Towards Open Vocabulary Learning: A Survey](https://mlsanity.com/paper/2306.15880/) (2023)
- [Open-Vocabulary Universal Image Segmentation with MaskCLIP](https://mlsanity.com/paper/2208.08984/) (2022)
- [Towards Label-free Scene Understanding by Vision Foundation Models](https://mlsanity.com/paper/2306.03899/) (2023)
- [Using a Waffle Iron for Automotive Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2301.10100/) (2023)
- [Better Call SAL: Towards Learning to Segment Anything in Lidar](https://mlsanity.com/paper/2403.13129/) (2024)
- [Three Pillars improving Vision Foundation Model Distillation for Lidar](https://mlsanity.com/paper/2310.17504/) (2023)
- [When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models](https://mlsanity.com/paper/2405.10255/) (2024)
- [CLIP-DIY: CLIP Dense Inference Yields Open-Vocabulary Semantic Segmentation For-Free](https://mlsanity.com/paper/2309.14289/) (2023)
- [Image Segmentation in Foundation Model Era: A Survey](https://mlsanity.com/paper/2408.12957/) (2024)
- [Open Vocabulary 3D Scene Understanding via Geometry Guided Self-Distillation](https://mlsanity.com/paper/2407.13362/) (2024)
- [Vision-Language Model for Object Detection and Segmentation: A Review and Evaluation](https://mlsanity.com/paper/2504.09480/) (2025)
- [LeAP: Consistent multi-domain 3D labeling using Foundation Models](https://mlsanity.com/paper/2502.03901/) (2025)
- [AdaCo: Overcoming Visual Foundation Model Noise in 3D Semantic Segmentation via Adaptive Label Correction](https://mlsanity.com/paper/2412.18255/) (2024)

## Built on by (most cited first, 5 of 5)

- [OccAny: Generalized Unconstrained Urban 3D Occupancy](https://mlsanity.com/paper/2603.23502/) (2026)
- [Depth-Only Open-Vocabulary 3D Semantic Segmentation For Privacy-Preserving Robotic Applications](https://mlsanity.com/paper/2607.00978/) (2026)
- [Annotation Free Spacecraft Detection and Segmentation using Vision Language Models](https://mlsanity.com/paper/2602.04699/) (2026)
- [IGLOSS: Image Generation for Lidar Open-vocabulary Semantic Segmentation](https://mlsanity.com/paper/2604.01361/) (2026)
- [Lang3DSeg: Annotation-Free Open-Vocabulary 3D Segmentation with Point Transformers](https://mlsanity.com/paper/2610.00855/) (2026)

## Abstract (arXiv)

We study the use of image-based Vision-Language Models (VLMs) for open-vocabulary segmentation of lidar scans in driving settings. Classically, image semantics can be back-projected onto 3D point clouds. Yet, resulting point labels are noisy and sparse. We consolidate these labels to enforce both spatio-temporal consistency and robustness to image-level augmentations. We then train a 3D network based on these refined labels. This simple method, called LOSC, outperforms the SOTA of zero-shot open-vocabulary semantic and panoptic segmentation on both nuScenes and SemanticKITTI, with significant margins. Code is available at https://github.com/valeoai/LOSC.