# GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervision

arXiv: https://arxiv.org/abs/2405.10591 · HTML: https://mlsanity.com/paper/2405.10591/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 51 references that are in the map.
- First posted on arXiv 17 May 2024.

## Builds on (most cited first, 30 of 32)

- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2302.07817/) (2023)
- [MonoScene: Monocular 3D Semantic Scene Completion](https://mlsanity.com/paper/2112.00726/) (2021)
- [BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection](https://mlsanity.com/paper/2203.17054/) (2022)
- [Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving](https://mlsanity.com/paper/2304.14365/) (2023)
- [SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving](https://mlsanity.com/paper/2303.09551/) (2023)
- [VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion](https://mlsanity.com/paper/2302.12251/) (2023)
- [OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2304.05316/) (2023)
- [OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception](https://mlsanity.com/paper/2303.03991/) (2023)
- [Mask2Former for Video Instance Segmentation](https://mlsanity.com/paper/2112.10764/) (2021)
- [FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformation](https://mlsanity.com/paper/2307.01492/) (2023)
- [SelfOcc: Self-Supervised Vision-Based 3D Occupancy Prediction](https://mlsanity.com/paper/2311.12754/) (2023)
- [RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervision](https://mlsanity.com/paper/2309.09502/) (2023)
- [Vision-Centric BEV Perception: A Survey](https://mlsanity.com/paper/2208.02797/) (2022)
- [PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentation](https://mlsanity.com/paper/2306.10013/) (2023)
- [FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugin](https://mlsanity.com/paper/2311.12058/) (2023)
- [COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction](https://mlsanity.com/paper/2312.01919/) (2023)
- [SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Prediction](https://mlsanity.com/paper/2404.09502/) (2024)
- [Differentiable Raycasting for Self-supervised Occupancy Forecasting](https://mlsanity.com/paper/2210.01917/) (2022)
- [Point Cloud Forecasting as a Proxy for 4D Occupancy Forecasting](https://mlsanity.com/paper/2302.13130/) (2023)
- [Occupancy Flow Fields for Motion Forecasting in Autonomous Driving](https://mlsanity.com/paper/2203.03875/) (2022)
- [OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queries](https://mlsanity.com/paper/2312.03774/) (2023)
- [Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving](https://mlsanity.com/paper/2308.01471/) (2023)
- [RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillation](https://mlsanity.com/paper/2312.11829/) (2023)
- [UniOcc: Unifying Vision-Centric 3D Occupancy Prediction with Geometric and Semantic Rendering](https://mlsanity.com/paper/2306.09117/) (2023)
- [UniScene: Multi-Camera Unified Pre-training via 3D Scene Reconstruction for Autonomous Driving](https://mlsanity.com/paper/2305.18829/) (2023)

## Built on by (most cited first, 21 of 21)

- [GaussianFlowOcc: Sparse and Weakly Supervised Occupancy Estimation using Gaussian Splatting and Temporal Flow](https://mlsanity.com/paper/2502.17288/) (2025)
- [LangOcc: Self-Supervised Open Vocabulary Occupancy Estimation via Volume Rendering](https://mlsanity.com/paper/2407.17310/) (2024)
- [Deep Height Decoupling for Precise Vision-based 3D Occupancy Prediction](https://mlsanity.com/paper/2409.07972/) (2024)
- [ShelfOcc: Native 3D Supervision beyond LiDAR for Vision-Based Occupancy Estimation](https://mlsanity.com/paper/2511.15396/) (2025)
- [OccCylindrical: Multi-Modal Fusion with Cylindrical Representation for 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2505.03284/) (2025)
- [4D-ROLLS: 4D Radar Occupancy Learning via LiDAR Supervision](https://mlsanity.com/paper/2505.13905/) (2025)
- [MetaOcc: Spatio-Temporal Fusion of Surround-View 4D Radar and Camera for 3D Occupancy Prediction with Dual Training Strategies](https://mlsanity.com/paper/2501.15384/) (2025)
- [SGFormer: Satellite-Ground Fusion for 3D Semantic Scene Completion](https://mlsanity.com/paper/2503.16825/) (2025)
- [See through the Dark: Learning Illumination-affined Representations for Nighttime Occupancy Prediction](https://mlsanity.com/paper/2505.20641/) (2025)
- [GraphGSOcc: Semantic-Geometric Graph Transformer with Dynamic-Static Decoupling for 3D Gaussian Splatting-based Occupancy Prediction](https://mlsanity.com/paper/2506.14825/) (2025)
- [Easy3D-Labels: Supervising Semantic Occupancy Estimation with 3D Pseudo-Labels for Automotive Perception](https://mlsanity.com/paper/2509.26087/) (2025)
- [Collaborative Learning of Local 3D Occupancy Prediction and Versatile Global Occupancy Mapping](https://mlsanity.com/paper/2504.13596/) (2025)
- [LidarPainter: One-Step Away From Any Lidar View To Novel Guidance](https://mlsanity.com/paper/2507.12114/) (2025)
- [PTQAT: A Hybrid Parameter-Efficient Quantization Algorithm for 3D Perception Tasks](https://mlsanity.com/paper/2508.10557/) (2025)
- [PFDepth: Heterogeneous Pinhole-Fisheye Joint Depth Estimation via Distortion-aware Gaussian-Splatted Volumetric Fusion](https://mlsanity.com/paper/2509.26008/) (2025)
- [SuperOcc: Toward Cohesive Temporal Modeling for Superquadric-based 3D Occupancy Prediction](https://mlsanity.com/paper/2601.15644/) (2026)
- [Streaming Dense Voxel Representations for 3D Occupancy Prediction](https://mlsanity.com/paper/2503.22087/) (2025)
- [TACOcc:Target-Adaptive Cross-Modal Fusion with Volume Rendering for 3D Semantic Occupancy](https://mlsanity.com/paper/2505.12693/) (2025)
- [FMOcc: TPV-Driven Flow Matching for 3D Occupancy Prediction with Selective State Space Model](https://mlsanity.com/paper/2507.02250/) (2025)
- [Adaptive Multi-Scale Channel-Spatial Attention Aggregation Framework for 3D Indoor Semantic Scene Completion Toward Assisting Visually Impaired](https://mlsanity.com/paper/2602.16385/) (2026)
- [SparseOcc++: Geometry-Aware Sparse Latent Representation for Semantic Occupancy Prediction](https://mlsanity.com/paper/2607.04732/) (2026)

## Abstract (arXiv)

3D occupancy perception holds a pivotal role in recent vision-centric autonomous driving systems by converting surround-view images into integrated geometric and semantic representations within dense 3D grids. Nevertheless, current models still encounter two main challenges: modeling depth accurately in the 2D-3D view transformation stage, and overcoming the lack of generalizability issues due to sparse LiDAR supervision. To address these issues, this paper presents GEOcc, a Geometric-Enhanced Occupancy network tailored for vision-only surround-view perception. Our approach is three-fold: 1) Integration of explicit lift-based depth prediction and implicit projection-based transformers for depth modeling, enhancing the density and robustness of view transformation. 2) Utilization of mask-based encoder-decoder architecture for fine-grained semantic predictions; 3) Adoption of context-aware self-training loss functions in the pertaining stage to complement LiDAR supervision, involving the re-rendering of 2D depth maps from 3D occupancy features and leveraging image reconstruction loss to obtain denser depth supervision besides sparse LiDAR ground-truths. Our approach achieves State-Of-The-Art performance on the Occ3D-nuScenes dataset with the least image resolution needed and the most weightless image backbone compared with current models, marking an improvement of 3.3% due to our proposed contributions. Comprehensive experimentation also demonstrates the consistent superiority of our method over baselines and alternative approaches.