# LightOcc: Lightweight Spatial Embedding for Efficient Vision-based 3D Occupancy Prediction

arXiv: https://arxiv.org/abs/2412.05976 · HTML: https://mlsanity.com/paper/2412.05976/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 61 references that are in the map.
- First posted on arXiv 8 Dec 2024.

## Builds on (most cited first, 30 of 48)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2302.07817/) (2023)
- [MonoScene: Monocular 3D Semantic Scene Completion](https://mlsanity.com/paper/2112.00726/) (2021)
- [BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection](https://mlsanity.com/paper/2203.17054/) (2022)
- [Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving](https://mlsanity.com/paper/2304.14365/) (2023)
- [SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving](https://mlsanity.com/paper/2303.09551/) (2023)
- [VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion](https://mlsanity.com/paper/2302.12251/) (2023)
- [BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision](https://mlsanity.com/paper/2211.10439/) (2022)
- [OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2304.05316/) (2023)
- [OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception](https://mlsanity.com/paper/2303.03991/) (2023)
- [Atlas: End-to-End 3D Scene Reconstruction from Posed Images](https://mlsanity.com/paper/2003.10432/) (2020)
- [FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformation](https://mlsanity.com/paper/2307.01492/) (2023)
- [GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2405.17429/) (2024)
- [PolarFormer: Multi-camera 3D Object Detection with Polar Transformer](https://mlsanity.com/paper/2206.15398/) (2022)
- [SelfOcc: Self-Supervised Vision-Based 3D Occupancy Prediction](https://mlsanity.com/paper/2311.12754/) (2023)
- [RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervision](https://mlsanity.com/paper/2309.09502/) (2023)
- [Vision-Centric BEV Perception: A Survey](https://mlsanity.com/paper/2208.02797/) (2022)
- [PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentation](https://mlsanity.com/paper/2306.10013/) (2023)
- [3D Object Detection for Autonomous Driving: A Survey](https://mlsanity.com/paper/2106.10823/) (2021)
- [FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugin](https://mlsanity.com/paper/2311.12058/) (2023)
- [COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction](https://mlsanity.com/paper/2312.01919/) (2023)
- [Symphonize 3D Semantic Scene Completion with Contextual Instance Queries](https://mlsanity.com/paper/2306.15670/) (2023)
- [GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Prediction](https://mlsanity.com/paper/2412.04384/) (2024)
- [MobileStereoNet: Towards Lightweight Deep Networks for Stereo Matching](https://mlsanity.com/paper/2108.09770/) (2021)

## Built on by (most cited first, 6 of 6)

- [ResWorld: Temporal Residual World Model for End-to-End Autonomous Driving](https://mlsanity.com/paper/2602.10884/) (2026)
- [Semantic Causality-Aware Vision-Based 3D Occupancy Prediction](https://mlsanity.com/paper/2509.08388/) (2025)
- [DA-Occ: Direction-Aware 2D Convolution for Efficient and Geometry-Preserving 3D Occupancy Prediction in Autonomous Driving](https://mlsanity.com/paper/2507.23599/) (2025)
- [SuperOcc: Toward Cohesive Temporal Modeling for Superquadric-based 3D Occupancy Prediction](https://mlsanity.com/paper/2601.15644/) (2026)
- [OccLE: Label-Efficient 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2505.20617/) (2025)
- [Adaptive Multi-Scale Channel-Spatial Attention Aggregation Framework for 3D Indoor Semantic Scene Completion Toward Assisting Visually Impaired](https://mlsanity.com/paper/2602.16385/) (2026)

## Abstract (arXiv)

Occupancy prediction has garnered increasing attention in recent years for its comprehensive fine-grained environmental representation and strong generalization to open-set objects. Nevertheless, mainstream occupancy prediction methods employ cumbersome voxel features as the scene representation, incurring substantial overheads in both memory and computation. When comparing the occupancy distribution in each spatial dimension, we find that the information entropy of the height dimension is much lower than the other two dimensions that constitute the Bird's Eye View (BEV) plane, which indicates that the height distribution of occupancy is easier to learn and predict. Accordingly, we propose Lightweight Spatial Embedding that can represent complete height information in a more compact way than voxel features, thus significantly enhancing its deployability. First, Single-Channel Occupancy is sampled from the multi-view depth distributions, which is then processed by Spatial-to-Channel mechanism to extract Lightweight Spatial Embeddings of different views by 2D convolution. These embeddings will interact with each other through the Lightweight Cross-View Interaction module to obtain the Unified Embedding, which can directly supplement BEV features with height information. Furthermore, we extract Edge-aware Spatial Embedding and apply Geometric Supervision on Spatial Embeddings, aiming to enhance their capability to represent spatial information. We also propose BEV-CutMix, a feature-level data augmentation strategy, to increase the diversity of the driving scenes. We integrate these innovative components into a pure 2D convolutional model, namely LightOcc. Sufficient experimental results show that LightOcc achieves state-of-the-art performance on multiple benchmarks while demonstrating significant efficiency advantages.