M2-Occ: Resilient 3D Semantic Occupancy Prediction for Autonomous Driving with Incomplete Camera Inputs
Not on the map. Why?
Abstract (arXiv)
Semantic occupancy prediction enables dense 3D geometric and semantic understanding for autonomous driving. However, existing camera-based approaches implicitly assume complete surround-view observations, an assumption that rarely holds in real-world deployment due to occlusion, hardware malfunction, or communication failures. We study semantic occupancy prediction under incomplete multi-camera inputs and introduce M2-Occ, a framework designed to preserve geometric structure and semantic coherence when views are missing. M2-Occ addresses two complementary challenges. First, a Multi-view Masked Reconstruction (MMR) module leverages the spatial overlap among neighboring cameras to recover missing-view representations directly in the feature space. Second, a Feature Memory Module (FMM) introduces a learnable memory bank that stores class-level semantic prototypes. By retrieving and integrating these global priors, the FMM refines ambiguous voxel features, ensuring semantic consistency even when observational evidence is incomplete. We introduce a systematic missing-view evaluation protocol on the nuScenes-based SurroundOcc benchmark, encompassing both deterministic single-view failures and stochastic multi-view dropout scenarios. Under the safety-critical missing back-view setting, M2-Occ improves the IoU by 4.36%. As the number of missing cameras increases, the robustness gap further widens; for instance, under the setting with five missing views, our method boosts the IoU by 6.67%. These gains are achieved without compromising full-view performance. The source code will be publicly released at https://github.com/qixi7up/M2-Occ.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 29 of its 35 references that are in the map. First posted on arXiv 10 Mar 2026.
Builds on
29 of 35 references.
References · 29
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2016Feature Pyramid Networks for Object Detectioncited by 5,036
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
- 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
- 2016Semantic Scene Completion from a Single Depth Imagecited by 472
- 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
- 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
- 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
- 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
- 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
17 more
- 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 293
- 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 261
- 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 223
- 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 221
- 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 188
- 2021BEVT: BERT Pretraining of Video Transformerscited by 169
- 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 96
- 2024POP-3D: Open-Vocabulary 3D Occupancy Prediction from Imagescited by 52
- 2024OccFusion: Multi-Sensor Fusion Framework for 3D Semantic Occupancy Predictioncited by 32
- 2023UniBEV: Multi-modal 3D Object Detection with Uniform BEV Encoders for Robustness against Missing Sensor Modalitiescited by 29
- 2025UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Drivingcited by 28
- 2025STCOcc: Sparse Spatial-Temporal Cascade Renovation for 3D Occupancy and Scene Flow Predictioncited by 23
- 2022BEV-MAE: Bird's Eye View Masked Autoencoders for Point Cloud Pre-training in Autonomous Driving Scenarioscited by 21
- 2025QuadricFormer: Scene as Superquadrics for 3D Semantic Occupancy Predictioncited by 20
- 2023M-BEV: Masked BEV Perception for Robust Autonomous Drivingcited by 19
- 2025SafeMap: Robust HD Map Construction from Incomplete Observationscited by 14
- 2025MS-Occ: Multi-Stage LiDAR-Camera Fusion for 3D Semantic Occupancy Predictioncited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.