mlsanityData to 14 Oct 2026

Fully Sparse 3D Occupancy Prediction

arXiv 2312.17118 ↗First posted 28 Dec 2023cs.CVcited by 64

Not on the map. Why?

Abstract (arXiv)

Occupancy prediction plays a pivotal role in autonomous driving. Previous methods typically construct dense 3D volumes, neglecting the inherent sparsity of the scene and suffering from high computational costs. To bridge the gap, we introduce a novel fully sparse occupancy network, termed SparseOcc. SparseOcc initially reconstructs a sparse 3D representation from camera-only inputs and subsequently predicts semantic/instance occupancy from the 3D sparse representation by sparse queries. A mask-guided sparse sampling is designed to enable sparse queries to interact with 2D features in a fully sparse manner, thereby circumventing costly dense features or global attention. Additionally, we design a thoughtful ray-based evaluation metric, namely RayIoU, to solve the inconsistency penalty along the depth axis raised in traditional voxel-level mIoU criteria. SparseOcc demonstrates its effectiveness by achieving a RayIoU of 34.0, while maintaining a real-time inference speed of 17.3 FPS, with 7 history frames inputs. By incorporating more preceding frames to 15, SparseOcc continuously improves its performance to 35.1 RayIoU without bells and whistles.

Key facts. Cited by 64 papers in the map (data through 14 Oct 2026). Builds on 62 of its 67 references that are in the map. First posted on arXiv 28 Dec 2023.

Builds on

62 of 67 references.

References · 62

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  9. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  10. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  11. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  12. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
38 more
  1. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
  2. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  3. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  4. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  5. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  6. 2018Panoptic Segmentationcited by 802
  7. 2020Convolutional Occupancy Networkscited by 761
  8. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  9. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  10. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  11. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  12. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  13. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
  14. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
  15. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 352
  16. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 341
  17. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  18. 2016Dynamic Filter Networkscited by 319
  19. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
  20. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 315
  21. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 314
  22. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
  23. 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 303
  24. 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 289
  25. 2019STD: Sparse-to-Dense 3D Object Detector for Point Cloudcited by 282
  26. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  27. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 221
  28. 2023Scene as Occupancycited by 209
  29. 2020Atlas: End-to-End 3D Scene Reconstruction from Posed Imagescited by 206
  30. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 165
  31. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 162
  32. 2023SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videoscited by 150
  33. 2022PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmarkcited by 140
  34. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 128
  35. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  36. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 115
  37. 2021TransformerFusion: Monocular RGB Scene Reconstruction using Transformerscited by 106
  38. 2023OpenLane-V2: A Topology Reasoning Benchmark for Unified 3D HD Mappingcited by 83

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 26; 2026: 28.

64 papers.

Sort:

By area:

  1. 2024A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspectivecited by 37
  2. 2025Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Drivingcited by 29
  3. 2024Let Occ Flow: Self-Supervised 3D Occupancy Flow Predictioncited by 24
  4. 2025STCOcc: Sparse Spatial-Temporal Cascade Renovation for 3D Occupancy and Scene Flow Predictioncited by 23
  5. 2024ALOcc: Adaptive Lifting-Based 3D Semantic Occupancy and Cost Volume-Based Flow Predictionscited by 20
  6. 2025GaussRender: Learning 3D Occupancy with Gaussian Renderingcited by 18
  7. 2025PRIX: Learning to Plan from Raw Pixels for End-to-End Autonomous Drivingcited by 16
  8. 2024DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Predictioncited by 15
  9. 2024VisionPAD: A Vision-Centric Pre-training Paradigm for Autonomous Drivingcited by 15
  10. 2025DrivePI: Spatial-aware 4D MLLM for Unified Autonomous Driving Understanding, Perception, Prediction and Planningcited by 15
  11. 2024Offboard Occupancy Refinement with Hybrid Propagation for Autonomous Drivingcited by 12
  12. 2025Rethinking Temporal Fusion with a Unified Gradient Descent View for 3D Semantic Occupancy Predictioncited by 9
38 more
  1. 2025Occupancy Learning with Spatiotemporal Memorycited by 9
  2. 2026Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Drivingcited by 9
  3. 2024EFFOcc: Learning Efficient Occupancy Networks from Minimal Labels for Autonomous Drivingcited by 8
  4. 2025Semantic Causality-Aware Vision-Based 3D Occupancy Predictioncited by 8
  5. 2024LightOcc: Lightweight Spatial Embedding for Efficient Vision-based 3D Occupancy Predictioncited by 6
  6. 2025Foundation Models for Autonomous Driving Perception: A Survey Through Core Capabilitiescited by 6
  7. 2024EPRecon: An Efficient Framework for Real-Time Panoptic 3D Reconstruction from Monocular Videocited by 4
  8. 2026SparseOccVLA: Bridging Occupancy and Vision-Language Models via Sparse Queries for Unified 4D Scene Understanding and Planningcited by 4
  9. 2026Enhancing Indoor Occupancy Prediction via Sparse Query-Based Multi-Level Consistent Knowledge Distillationcited by 4
  10. 2026FlowAD: Ego-Scene Interactive Modeling for Autonomous Drivingcited by 4
  11. 2025Event-aided Semantic Scene Completioncited by 3
  12. 2025See through the Dark: Learning Illumination-affined Representations for Nighttime Occupancy Predictioncited by 3
  13. 2025UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNscited by 3
  14. 2025MinkOcc: Towards real-time label-efficient semantic occupancy predictioncited by 2
  15. 2025Camera-Only 3D Panoptic Scene Completion for Autonomous Driving through Differentiable Object Shapescited by 2
  16. 2025Out-of-Distribution Semantic Occupancy Predictioncited by 2
  17. 2025QueryOcc: Query-based Self-Supervision for 3D Semantic Occupancycited by 2
  18. 2026The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Modelscited by 2
  19. 2026Geometry-Grounded Unified 3D Perception for Autonomous Drivingcited by 2
  20. 2025TGP: Two-modal occupancy prediction with 3D Gaussian and sparse points for 3D Environment Awarenesscited by 1
  21. 2025S2GO: Streaming Sparse Gaussian Occupancy Predictioncited by 1
  22. 2025Boosting Multi-View Indoor 3D Object Detection via Adaptive 3D Volume Constructioncited by 1
  23. 2025DA-Occ: Direction-Aware 2D Convolution for Efficient and Geometry-Preserving 3D Occupancy Prediction in Autonomous Drivingcited by 1
  24. 2026SuperOcc: Toward Cohesive Temporal Modeling for Superquadric-based 3D Occupancy Predictioncited by 1
  25. 2026SelfOccFlow: Towards end-to-end self-supervised 3D Occupancy Flow predictioncited by 1
  26. 2026An Instance-Centric Panoptic Occupancy Prediction Benchmark for Autonomous Drivingcited by 1
  27. 2026VGGT-Occ: Geometry-Grounded and Density-Aware Gated Fusion for 3D Occupancy Predictioncited by 1
  28. 2026Sparse-Aware Vector Quantization for Bandwidth-Efficient Collaborative 3D Semantic Occupancy Predictioncited by 1
  29. 2024BDC-Occ: Binarized Deep Convolution Unit For Binarized Occupancy Networkcited by 0
  30. 2025Deflickering Vision-Based Occupancy Networks through Lightweight Spatio-Temporal Correlationcited by 0
  31. 2025InstanceBEV: Unifying Instance and BEV Representation for 3D Panoptic Segmentationcited by 0
  32. 2025Object Concepts Emerge from Motioncited by 0
  33. 2025FMOcc: TPV-Driven Flow Matching for 3D Occupancy Prediction with Selective State Space Modelcited by 0
  34. 2025From Binary to Semantic: Utilizing Large-Scale Binary Occupancy Data for 3D Semantic Occupancy Predictioncited by 0
  35. 2025HyGE-Occ: Hybrid View-Transformation with 3D Gaussian and Edge Priors for 3D Panoptic Occupancy Predictioncited by 0
  36. 2026SUG-Occ: Explicit Semantics and Uncertainty Guided Sparse Learning for Efficient 3D Occupancy Predictioncited by 0
  37. 2026FreeOcc: Training-free Panoptic Occupancy Prediction via Foundation Modelscited by 0
  38. 2026PLAF: Pixel-wise Language-Aligned Feature Extraction for Efficient 3D Scene Understandingcited by 0

50 most cited of 64 All 64 (JSON)