# VLScene: Vision-Language Guidance Distillation for Camera-Based 3D Semantic Scene Completion

arXiv: https://arxiv.org/abs/2503.06219 · HTML: https://mlsanity.com/paper/2503.06219/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 8 Mar 2025.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Improved Knowledge Distillation via Teacher Assistant](https://mlsanity.com/paper/1902.03393/) (2019)
- [Scaling Open-Vocabulary Image Segmentation with Image-Level Labels](https://mlsanity.com/paper/2112.12143/) (2021)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2302.07817/) (2023)
- [Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation](https://mlsanity.com/paper/2011.10033/) (2020)
- [MonoScene: Monocular 3D Semantic Scene Completion](https://mlsanity.com/paper/2112.00726/) (2021)
- [VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion](https://mlsanity.com/paper/2302.12251/) (2023)
- [OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2304.05316/) (2023)
- [Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion](https://mlsanity.com/paper/2012.03762/) (2020)
- [CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP](https://mlsanity.com/paper/2301.04926/) (2023)
- [Anisotropic Convolutional Networks for 3D Semantic Scene Completion](https://mlsanity.com/paper/2004.02122/) (2020)
- [RegionPLC: Regional Point-Language Contrastive Learning for Open-World 3D Scene Understanding](https://mlsanity.com/paper/2304.00962/) (2023)
- [Symphonize 3D Semantic Scene Completion with Contextual Instance Queries](https://mlsanity.com/paper/2306.15670/) (2023)
- [NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Space](https://mlsanity.com/paper/2309.14616/) (2023)
- [SCPNet: Semantic Scene Completion on Point Cloud](https://mlsanity.com/paper/2303.06884/) (2023)
- [Semantic Scene Completion using Local Deep Implicit Functions on LiDAR Data](https://mlsanity.com/paper/2011.09141/) (2020)
- [Efficient Semantic Scene Completion Network with Spatial Group Convolution](https://mlsanity.com/paper/1907.05091/) (2019)
- [RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completion](https://mlsanity.com/paper/1903.00620/) (2019)
- [Semantic Scene Completion via Integrating Instances and Scene in-the-Loop](https://mlsanity.com/paper/2104.03640/) (2021)
- [MonoOcc: Digging into Monocular Semantic Occupancy Prediction](https://mlsanity.com/paper/2403.08766/) (2024)
- [Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillation](https://mlsanity.com/paper/2404.11958/) (2024)
- [View-volume Network for Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1806.05361/) (2018)
- [HEAD: HEtero-Assists Distillation for Heterogeneous Object Detectors](https://mlsanity.com/paper/2207.05345/) (2022)

## Built on by (most cited first, 6 of 6)

- [VoxDet: Rethinking 3D Semantic Occupancy Prediction as Dense Object Detection](https://mlsanity.com/paper/2506.04623/) (2025)
- [Enhancing 3D Semantic Scene Completion with a Refinement Module](https://mlsanity.com/paper/2512.18363/) (2025)
- [Unleashing Semantic and Geometric Priors for 3D Scene Completion](https://mlsanity.com/paper/2508.13601/) (2025)
- [Sparsity-Aware Voxel Attention and Foreground Modulation for 3D Semantic Scene Completion](https://mlsanity.com/paper/2604.05780/) (2026)
- [RayLift: Lifting Complementary Ray-Wise Evidence with 3D Geometry Priors for Semantic Scene Completion](https://mlsanity.com/paper/2608.08476/) (2026)
- [ProjFormer: Point Cloud Completion via Geometric-Projective Transformer and Cross-Modal Semantic Constraints](https://mlsanity.com/paper/2608.15104/) (2026)

## Abstract (arXiv)

Camera-based 3D semantic scene completion (SSC) provides dense geometric and semantic perception for autonomous driving. However, images provide limited information making the model susceptible to geometric ambiguity caused by occlusion and perspective distortion. Existing methods often lack explicit semantic modeling between objects, limiting their perception of 3D semantic context. To address these challenges, we propose a novel method VLScene: Vision-Language Guidance Distillation for Camera-based 3D Semantic Scene Completion. The key insight is to use the vision-language model to introduce high-level semantic priors to provide the object spatial context required for 3D scene understanding. Specifically, we design a vision-language guidance distillation process to enhance image features, which can effectively capture semantic knowledge from the surrounding environment and improve spatial context reasoning. In addition, we introduce a geometric-semantic sparse awareness mechanism to propagate geometric structures in the neighborhood and enhance semantic information through contextual sparse interactions. Experimental results demonstrate that VLScene achieves rank-1st performance on challenging benchmarks--SemanticKITTI and SSCBench-KITTI-360, yielding remarkably mIoU scores of 17.52 and 19.10, respectively.