# MonoOcc: Digging into Monocular Semantic Occupancy Prediction

arXiv: https://arxiv.org/abs/2403.08766 · HTML: https://mlsanity.com/paper/2403.08766/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 51 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Builds on (most cited first, 30 of 40)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions](https://mlsanity.com/paper/2211.05778/) (2022)
- [Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2302.07817/) (2023)
- [Learning by Cheating](https://mlsanity.com/paper/1912.12294/) (2019)
- [MonoScene: Monocular 3D Semantic Scene Completion](https://mlsanity.com/paper/2112.00726/) (2021)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving](https://mlsanity.com/paper/2303.09551/) (2023)
- [VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion](https://mlsanity.com/paper/2302.12251/) (2023)
- [OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2304.05316/) (2023)
- [Unifying Flow, Stereo and Depth Estimation](https://mlsanity.com/paper/2211.05783/) (2022)
- [Cross-view Transformers for real-time Map-view Semantic Segmentation](https://mlsanity.com/paper/2205.02833/) (2022)
- [FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras](https://mlsanity.com/paper/2104.10490/) (2021)
- [Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion](https://mlsanity.com/paper/2012.03762/) (2020)
- [LMSCNet: Lightweight Multiscale 3D Semantic Completion](https://mlsanity.com/paper/2008.10559/) (2020)
- [Anisotropic Convolutional Networks for 3D Semantic Scene Completion](https://mlsanity.com/paper/2004.02122/) (2020)
- [3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior](https://mlsanity.com/paper/2003.14052/) (2020)
- [Translating Images into Maps](https://mlsanity.com/paper/2110.00966/) (2021)
- [RigNet: Repetitive Image Guided Network for Depth Completion](https://mlsanity.com/paper/2107.13802/) (2021)

## Built on by (most cited first, 30 of 44)

- [World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model](https://mlsanity.com/paper/2507.00603/) (2025)
- [DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Model](https://mlsanity.com/paper/2410.10429/) (2024)
- [Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Models](https://mlsanity.com/paper/2505.23757/) (2025)
- [Context and Geometry Aware Voxel Transformer for Semantic Scene Completion](https://mlsanity.com/paper/2405.13675/) (2024)
- [Tri-Perspective View Decomposition for Geometry-Aware Depth Completion](https://mlsanity.com/paper/2403.15008/) (2024)
- [A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspective](https://mlsanity.com/paper/2405.05173/) (2024)
- [Vision-based 3D occupancy prediction in autonomous driving: a review and outlook](https://mlsanity.com/paper/2405.02595/) (2024)
- [Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving](https://mlsanity.com/paper/2502.07309/) (2025)
- [AVD2: Accident Video Diffusion for Accident Video Description](https://mlsanity.com/paper/2502.14801/) (2025)
- [LiCROcc: Teach Radar for Accurate Semantic Occupancy Prediction using LiDAR and Camera](https://mlsanity.com/paper/2407.16197/) (2024)
- [Language Driven Occupancy Prediction](https://mlsanity.com/paper/2411.16072/) (2024)
- [Chameleon: Fast-slow Neuro-symbolic Lane Topology Extraction](https://mlsanity.com/paper/2503.07485/) (2025)
- [Challenger: Affordable Adversarial Driving Video Generation](https://mlsanity.com/paper/2505.15880/) (2025)
- [GaussianFormer3D: Multi-Modal Gaussian-based Semantic Occupancy Prediction with 3D Deformable Attention](https://mlsanity.com/paper/2505.10685/) (2025)
- [VoxDet: Rethinking 3D Semantic Occupancy Prediction as Dense Object Detection](https://mlsanity.com/paper/2506.04623/) (2025)
- [WildOcc: A Benchmark for Off-Road 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2410.15792/) (2024)
- [Skip Mamba Diffusion for Monocular 3D Semantic Scene Completion](https://mlsanity.com/paper/2501.07260/) (2025)
- [Reusing Attention for One-stage Lane Topology Understanding](https://mlsanity.com/paper/2507.17617/) (2025)
- [OccTENS: 3D Occupancy World Model via Temporal Next-Scale Prediction](https://mlsanity.com/paper/2509.03887/) (2025)
- [LOMA: Language-assisted Semantic Occupancy Network via Triplane Mamba](https://mlsanity.com/paper/2412.08388/) (2024)
- [ViPOcc: Leveraging Visual Priors from Vision Foundation Models for Single-View 3D Occupancy Prediction](https://mlsanity.com/paper/2412.11210/) (2024)
- [L2COcc: Lightweight Camera-Centric Semantic Scene Completion via Distillation of LiDAR Model](https://mlsanity.com/paper/2503.12369/) (2025)
- [Disentangling Instance and Scene Contexts for 3D Semantic Scene Completion](https://mlsanity.com/paper/2507.08555/) (2025)
- [VisHall3D: Monocular Semantic Scene Completion from Reconstructing the Visible Regions to Hallucinating the Invisible Regions](https://mlsanity.com/paper/2507.19188/) (2025)
- [VLScene: Vision-Language Guidance Distillation for Camera-Based 3D Semantic Scene Completion](https://mlsanity.com/paper/2503.06219/) (2025)
- [Three Cars Approaching within 100m! Enhancing Distant Geometry by Tri-Axis Voxel Scanning for Camera-based Semantic Scene Completion](https://mlsanity.com/paper/2411.16129/) (2024)
- [PosePilot: Steering Camera Pose for Generative World Models with Self-supervised Depth](https://mlsanity.com/paper/2505.01729/) (2025)
- [Enhancing Indoor Occupancy Prediction via Sparse Query-Based Multi-Level Consistent Knowledge Distillation](https://mlsanity.com/paper/2602.02318/) (2026)
- [ET-Former: Efficient Triplane Deformable Attention for 3D Semantic Scene Completion From Monocular Camera](https://mlsanity.com/paper/2410.11019/) (2024)
- [Learning Temporal 3D Semantic Scene Completion via Optical Flow Guidance](https://mlsanity.com/paper/2502.14520/) (2025)

## Abstract (arXiv)

Monocular Semantic Occupancy Prediction aims to infer the complete 3D geometry and semantic information of scenes from only 2D images. It has garnered significant attention, particularly due to its potential to enhance the 3D perception of autonomous vehicles. However, existing methods rely on a complex cascaded framework with relatively limited information to restore 3D scenes, including a dependency on supervision solely on the whole network's output, single-frame input, and the utilization of a small backbone. These challenges, in turn, hinder the optimization of the framework and yield inferior prediction results, particularly concerning smaller and long-tailed objects. To address these issues, we propose MonoOcc. In particular, we (i) improve the monocular occupancy prediction framework by proposing an auxiliary semantic loss as supervision to the shallow layers of the framework and an image-conditioned cross-attention module to refine voxel features with visual clues, and (ii) employ a distillation module that transfers temporal information and richer knowledge from a larger image backbone to the monocular semantic occupancy prediction framework with low cost of hardware. With these advantages, our method yields state-of-the-art performance on the camera-based SemanticKITTI Scene Completion benchmark. Codes and models can be accessed at https://github.com/ucaszyp/MonoOcc