mlsanityData to 14 Oct 2026

OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction

arXiv 2304.05316 ↗First posted 11 Apr 2023cs.CVcited by 293

Not on the map. Why?

Cited by 25 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The vision-based perception for autonomous driving has undergone a transformation from the bird-eye-view (BEV) representations to the 3D semantic occupancy. Compared with the BEV planes, the 3D semantic occupancy further provides structural information along the vertical direction. This paper presents OccFormer, a dual-path transformer network to effectively process the 3D volume for semantic occupancy prediction. OccFormer achieves a long-range, dynamic, and efficient encoding of the camera-generated 3D voxel features. It is obtained by decomposing the heavy 3D processing into the local and global transformer pathways along the horizontal plane. For the occupancy decoder, we adapt the vanilla Mask2Former for 3D semantic occupancy by proposing preserve-pooling and class-guided sampling, which notably mitigate the sparsity and class imbalance. Experimental results demonstrate that OccFormer significantly outperforms existing methods for semantic scene completion on SemanticKITTI dataset and for LiDAR semantic segmentation on nuScenes dataset. Code is available at urlhttps://github.com/zhangyp15/OccFormer.

Key facts. Cited by 293 papers in the map (data through 14 Oct 2026). Builds on 58 of its 65 references that are in the map. First posted on arXiv 11 Apr 2023.

Builds on

58 of 65 references.

References · 58

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  9. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  10. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  11. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  12. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
38 more
  1. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  2. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  3. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  4. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  5. 2021Video Swin Transformercited by 1,091
  6. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  7. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 941
  8. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  9. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  10. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  11. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  12. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  13. 2016Semantic Scene Completion from a Single Depth Imagecited by 472
  14. 2019PointRend: Image Segmentation as Renderingcited by 440
  15. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 423
  16. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
  17. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 389
  18. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
  19. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 327
  20. 2018Video Action Transformer Networkcited by 316
  21. 2021PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformerscited by 286
  22. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 275
  23. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  24. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  25. 2020SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Drivingcited by 226
  26. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 223
  27. 2021ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Biascited by 216
  28. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  29. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 188
  30. 2020Predicting Semantic Map Representations from Images using Pyramid Occupancy Networkscited by 175
  31. 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 166
  32. 2021(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Networkcited by 164
  33. 2020Anisotropic Convolutional Networks for 3D Semantic Scene Completioncited by 127
  34. 20203D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Priorcited by 126
  35. 2022BEVSegFormer: Bird's Eye View Semantic Segmentation From Arbitrary Camera Rigscited by 123
  36. 2021MPViT: Multi-Path Vision Transformer for Dense Predictioncited by 122
  37. 2022ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyondcited by 117
  38. 2020S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Cloudscited by 102

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 21; 2024: 94; 2025: 108; 2026: 70.

293 papers.

Sort:

By area:

  1. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  2. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 157
  3. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  4. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 128
  5. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 115
  6. 2023FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugincited by 103
  7. 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 96
  8. 2024SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Predictioncited by 96
  9. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 90
  10. 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 86
  11. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 78
  12. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 75
38 more
  1. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Drivingcited by 69
  2. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 67
  3. 2024DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Modelcited by 61
  4. 2025A Survey of World Models for Autonomous Drivingcited by 60
  5. 2024OPUS: Occupancy Prediction Using a Sparse Setcited by 56
  6. 2024FastOcc: Accelerating 3D Occupancy Prediction by Fusing the 2D Bird's-Eye View and Perspective Viewcited by 51
  7. 2024Context and Geometry Aware Voxel Transformer for Semantic Scene Completioncited by 50
  8. 2024GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splattingcited by 50
  9. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  10. 2024Co-Occ: Coupling Explicit Feature Fusion with Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy Predictioncited by 48
  11. 2024RenderWorld: World Model with Self-Supervised 3D Labelcited by 46
  12. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understandingcited by 46
  13. 2024MonoOcc: Digging into Monocular Semantic Occupancy Predictioncited by 44
  14. 2024Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillationcited by 43
  15. 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environmentscited by 41
  16. 2024Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Reviewcited by 41
  17. 2023Camera-based 3D Semantic Scene Completion with Sparse Guidance Networkcited by 40
  18. 2024Occupancy as Set of Pointscited by 38
  19. 2024ViewFormer: Exploring Spatiotemporal Modeling for Multi-View 3D Occupancy Perception via View-Guided Transformerscited by 37
  20. 2024A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspectivecited by 37
  21. 2023UniWorld: Autonomous Driving Pre-training via World Modelscited by 33
  22. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  23. 2024Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehiclescited by 33
  24. 2024RadarOcc: Robust 3D Occupancy Prediction with 4D Imaging Radarcited by 33
  25. 2024DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Featurescited by 33
  26. 2025OmniNWM: Omniscient Driving Navigation World Modelscited by 33
  27. 2023DepthSSC: Monocular 3D Semantic Scene Completion via Depth-Spatial Alignment and Voxel Adaptationcited by 32
  28. 2024OccFusion: Multi-Sensor Fusion Framework for 3D Semantic Occupancy Predictioncited by 32
  29. 2024Hierarchical Temporal Context Learning for Camera-based Semantic Scene Completioncited by 32
  30. 2024Omni-Scene: Omni-Gaussian Representation for Ego-Centric Sparse-View Scene Reconstructioncited by 32
  31. 2024OccMamba: Semantic Occupancy Prediction with State Space Modelscited by 30
  32. 2024OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Drivingcited by 29
  33. 2024Vision-based 3D occupancy prediction in autonomous driving: a review and outlookcited by 29
  34. 2025Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Modelscited by 28
  35. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunitiescited by 24
  36. 2024Let Occ Flow: Self-Supervised 3D Occupancy Flow Predictioncited by 24
  37. 2025GaussianFlowOcc: Sparse and Weakly Supervised Occupancy Estimation using Gaussian Splatting and Temporal Flowcited by 24
  38. 2023LiDAR-based 4D Occupancy Completion and Forecastingcited by 23

50 most cited of 293 All 293 (JSON)