mlsanityData to 14 Oct 2026

SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving

arXiv 2303.09551 ↗First posted 16 Mar 2023cs.CVcited by 316

Not on the map. Why?

Cited by 29 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

3D scene understanding plays a vital role in vision-based autonomous driving. While most existing methods focus on 3D object detection, they have difficulty describing real-world objects of arbitrary shapes and infinite classes. Towards a more comprehensive perception of a 3D scene, in this paper, we propose a SurroundOcc method to predict the 3D occupancy with multi-camera images. We first extract multi-scale features for each image and adopt spatial 2D-3D attention to lift them to the 3D volume space. Then we apply 3D convolutions to progressively upsample the volume features and impose supervision on multiple levels. To obtain dense occupancy prediction, we design a pipeline to generate dense occupancy ground truth without expansive occupancy annotations. Specifically, we fuse multi-frame LiDAR scans of dynamic objects and static scenes separately. Then we adopt Poisson Reconstruction to fill the holes and voxelize the mesh to get dense occupancy labels. Extensive experiments on nuScenes and SemanticKITTI datasets demonstrate the superiority of our method. Code and dataset are available at https://github.com/weiyithu/SurroundOcc

Key facts. Cited by 316 papers in the map (data through 14 Oct 2026). Builds on 59 of its 71 references that are in the map. First posted on arXiv 16 Mar 2023.

Builds on

59 of 71 references.

References · 59

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  3. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  5. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  6. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  7. 2017Deformable Convolutional Networkscited by 1,488
  8. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  9. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  10. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  11. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  12. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
38 more
  1. 2020Convolutional Occupancy Networkscited by 761
  2. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  3. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  4. 2020AdaBins: Depth Estimation using Adaptive Binscited by 601
  5. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  6. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  7. 2018Pyramid Stereo Matching Networkcited by 489
  8. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  9. 2016Semantic Scene Completion from a Single Depth Imagecited by 472
  10. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 457
  11. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
  12. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 423
  13. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
  14. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 389
  15. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 382
  16. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
  17. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 342
  18. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 336
  19. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 327
  20. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
  21. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 249
  22. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
  23. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  24. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  25. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 223
  26. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  27. 2020Atlas: End-to-End 3D Scene Reconstruction from Posed Imagescited by 206
  28. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 191
  29. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  30. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 188
  31. 2021(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Networkcited by 164
  32. 2020MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationshipscited by 153
  33. 2021RangeDet:In Defense of Range View for LiDAR-based 3D Object Detectioncited by 140
  34. 2019Accurate Monocular Object Detection via Color-Embedded 3D Reconstruction for Autonomous Drivingcited by 137
  35. 2017SurfaceNet: An End-to-end 3D Neural Network for Multiview Stereopsiscited by 128
  36. 2020Anisotropic Convolutional Networks for 3D Semantic Scene Completioncited by 127
  37. 20203D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Priorcited by 126
  38. 2020Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentationcited by 112

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 32; 2024: 101; 2025: 104; 2026: 79.

316 papers.

Sort:

By area:

  1. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  2. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 233
  3. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  4. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 157
  5. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 147
  6. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 128
  7. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 115
  8. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 106
  9. 2024textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Drivingcited by 105
  10. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Drivingcited by 99
  11. 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 96
  12. 2024SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Predictioncited by 96
38 more
  1. 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 86
  2. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 78
  3. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Drivingcited by 69
  4. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 67
  5. 2023Fully Sparse 3D Occupancy Predictioncited by 64
  6. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  7. 2024SCube: Instant Large-Scale Scene Reconstruction using VoxSplatscited by 60
  8. 2025A Survey of World Models for Autonomous Drivingcited by 60
  9. 2023PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Predictioncited by 52
  10. 2023Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applicationscited by 52
  11. 2024FastOcc: Accelerating 3D Occupancy Prediction by Fusing the 2D Bird's-Eye View and Perspective Viewcited by 51
  12. 2024Context and Geometry Aware Voxel Transformer for Semantic Scene Completioncited by 50
  13. 2024GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splattingcited by 50
  14. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  15. 2024Co-Occ: Coupling Explicit Feature Fusion with Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy Predictioncited by 48
  16. 2024RenderWorld: World Model with Self-Supervised 3D Labelcited by 46
  17. 2024EmbodiedOcc: Embodied 3D Occupancy Prediction for Vision-based Online Scene Understandingcited by 45
  18. 2024MonoOcc: Digging into Monocular Semantic Occupancy Predictioncited by 44
  19. 2024End-to-End Autonomous Driving through V2X Cooperationcited by 42
  20. 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environmentscited by 41
  21. 2024Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Drivingcited by 41
  22. 2023Camera-based 3D Semantic Scene Completion with Sparse Guidance Networkcited by 40
  23. 2024Occupancy as Set of Pointscited by 38
  24. 2024A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspectivecited by 37
  25. 2024GeoLRM: Geometry-Aware Large Reconstruction Model for High-Quality 3D Gaussian Generationcited by 36
  26. 2025TartanGround: A Large-Scale Dataset for Ground Robot Perception and Navigationcited by 36
  27. 2023UniWorld: Autonomous Driving Pre-training via World Modelscited by 33
  28. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  29. 2024Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehiclescited by 33
  30. 2024RadarOcc: Robust 3D Occupancy Prediction with 4D Imaging Radarcited by 33
  31. 2025OmniNWM: Omniscient Driving Navigation World Modelscited by 33
  32. 2023DepthSSC: Monocular 3D Semantic Scene Completion via Depth-Spatial Alignment and Voxel Adaptationcited by 32
  33. 2024OccFusion: Multi-Sensor Fusion Framework for 3D Semantic Occupancy Predictioncited by 32
  34. 2024Hierarchical Temporal Context Learning for Camera-based Semantic Scene Completioncited by 32
  35. 2024Omni-Scene: Omni-Gaussian Representation for Ego-Centric Sparse-View Scene Reconstructioncited by 32
  36. 2024Monocular Occupancy Prediction for Scalable Indoor Scenescited by 31
  37. 2024OccMamba: Semantic Occupancy Prediction with State Space Modelscited by 30
  38. 2024OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Drivingcited by 29

50 most cited of 316 All 316 (JSON)