mlsanityData to 7 Oct 2026

FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformation

arXiv 2307.01492 ↗First posted 4 Jul 2023cs.CV cs.ROcited by 162

On the map: End-to-end autonomous driving → Generative simulation for driving

Abstract (arXiv)

This technical report summarizes the winning solution for the 3D Occupancy Prediction Challenge, which is held in conjunction with the CVPR 2023 Workshop on End-to-End Autonomous Driving and CVPR 23 Workshop on Vision-Centric Autonomous Driving Workshop. Our proposed solution FB-OCC builds upon FB-BEV, a cutting-edge camera-based bird's-eye view perception design using forward-backward projection. On top of FB-BEV, we further study novel designs and optimization tailored to the 3D occupancy prediction task, including joint depth-semantic pre-training, joint voxel-BEV representation, model scaling up, and effective post-processing strategies. These designs and optimization result in a state-of-the-art mIoU score of 54.19% on the nuScenes dataset, ranking the 1st place in the challenge track. Code and models will be released at: https://github.com/NVlabs/FB-BEV.

Key facts. Cited by 162 papers in the map (data through 7 Oct 2026). Placed in Generative simulation for driving, one of 9 directions in End-to-end autonomous driving. Builds on 17 of its 19 references that are in the map. First posted on arXiv 4 Jul 2023.

Builds on

17 of 19 references.

References · 17

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2022A ConvNet for the 2020scited by 3,545
  5. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  6. 2022Planning-oriented Autonomous Drivingcited by 1,068
  7. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  8. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  9. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  10. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  11. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 479
  12. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
5 more
  1. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  2. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 218
  3. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 177
  4. 2019An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detectioncited by 163
  5. 2022M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representationcited by 155

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 52; 2025: 65; 2026: 35.

162 papers.

Sort:

By area:

  1. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
  2. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  3. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  4. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 155
  5. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  6. 2023FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugincited by 101
  7. 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 83
  8. 2024OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Drivingcited by 82
  9. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 66
  10. 2023Fully Sparse 3D Occupancy Predictioncited by 61
  11. 2024DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Modelcited by 58
  12. 2024OPUS: Occupancy Prediction Using a Sparse Setcited by 56
38 more
  1. 2024FastOcc: Accelerating 3D Occupancy Prediction by Fusing the 2D Bird's-Eye View and Perspective Viewcited by 51
  2. 2024GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splattingcited by 50
  3. 2024Co-Occ: Coupling Explicit Feature Fusion with Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy Predictioncited by 46
  4. 2024RenderWorld: World Model with Self-Supervised 3D Labelcited by 44
  5. 2024Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillationcited by 43
  6. 2024EmbodiedOcc: Embodied 3D Occupancy Prediction for Vision-based Online Scene Understandingcited by 43
  7. 2023Camera-based 3D Semantic Scene Completion with Sparse Guidance Networkcited by 40
  8. 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environmentscited by 39
  9. 2024A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspectivecited by 37
  10. 2023BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detectioncited by 35
  11. 2024Unleashing HyDRa: Hybrid Fusion, Depth Consistency and Radar for Unified 3D Perceptioncited by 35
  12. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  13. 2024Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehiclescited by 33
  14. 2024OccFusion: Multi-Sensor Fusion Framework for 3D Semantic Occupancy Predictioncited by 32
  15. 2024Vision-based 3D occupancy prediction in autonomous driving: a review and outlookcited by 29
  16. 2024Monocular Occupancy Prediction for Scalable Indoor Scenescited by 29
  17. 2025Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Drivingcited by 29
  18. 2024OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Drivingcited by 28
  19. 2023S4C: Self-Supervised Semantic Scene Completion with Neural Fieldscited by 27
  20. 2025Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Modelscited by 27
  21. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunitiescited by 24
  22. 2025GaussianFlowOcc: Sparse and Weakly Supervised Occupancy Estimation using Gaussian Splatting and Temporal Flowcited by 24
  23. 2024Let Occ Flow: Self-Supervised 3D Occupancy Flow Predictioncited by 23
  24. 2024OccFlowNet: Towards Self-supervised Occupancy Estimation via Differentiable Rendering and Occupancy Flowcited by 22
  25. 2024PreSight: Enhancing Autonomous Vehicle Perception with City-Scale NeRF Priorscited by 21
  26. 2024GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervisioncited by 21
  27. 2025STCOcc: Sparse Spatial-Temporal Cascade Renovation for 3D Occupancy and Scene Flow Predictioncited by 20
  28. 2025DVGT: Driving Visual Geometry Transformercited by 20
  29. 2024ALOcc: Adaptive Lifting-Based 3D Semantic Occupancy and Cost Volume-Based Flow Predictionscited by 19
  30. 2025QuadricFormer: Scene as Superquadrics for 3D Semantic Occupancy Predictioncited by 19
  31. 2024ProtoOcc: Accurate, Efficient 3D Occupancy Prediction Using Dual Branch Encoder-Prototype Query Decodercited by 17
  32. 2024An Efficient Occupancy World Model via Decoupled Dynamic Flow and Image-assisted Trainingcited by 17
  33. 2025GaussRender: Learning 3D Occupancy with Gaussian Renderingcited by 17
  34. 2024LangOcc: Self-Supervised Open Vocabulary Occupancy Estimation via Volume Renderingcited by 16
  35. 2024Deep Height Decoupling for Precise Vision-based 3D Occupancy Predictioncited by 16
  36. 2025ORV: 4D Occupancy-centric Robot Video Generationcited by 16
  37. 2025Lotus-2: Advancing Geometric Dense Prediction with Powerful Image Generative Modelcited by 16
  38. 2024DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Predictioncited by 15

50 most cited of 162 All 162 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Spatiotemporal Decoupling for Efficient Vision-Based Occupancy Forecasting
  2. 2023Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications
  3. 2025FMOcc: TPV-Driven Flow Matching for 3D Occupancy Prediction with Selective State Space Model
  4. 2024MambaOcc: Visual State Space Model for BEV-based Occupancy Prediction with Local Adaptive Reordering
  5. 2025OccProphet: Pushing Efficiency Frontier of Camera-Only 4D Occupancy Forecasting with Observer-Forecaster-Refiner Framework
  6. 2024FSF-Net: Enhance 4D Occupancy Forecasting with Coarse BEV Scene Flow for Autonomous Driving
  7. 2023LiDAR-based 4D Occupancy Completion and Forecasting
  8. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving