mlsanityData to 14 Oct 2026

MinkOcc: Towards real-time label-efficient semantic occupancy prediction

arXiv 2504.02270 ↗First posted 3 Apr 2025cs.CV cs.ROcited by 2

Not on the map. Why?

Abstract (arXiv)

Developing 3D semantic occupancy prediction models often relies on dense 3D annotations for supervised learning, a process that is both labor and resource-intensive, underscoring the need for label-efficient or even label-free approaches. To address this, we introduce MinkOcc, a multi-modal 3D semantic occupancy prediction framework for cameras and LiDARs that proposes a two-step semi-supervised training procedure. Here, a small dataset of explicitly 3D annotations warm-starts the training process; then, the supervision is continued by simpler-to-annotate accumulated LiDAR sweeps and images -- semantically labelled through vision foundational models. MinkOcc effectively utilizes these sensor-rich supervisory cues and reduces reliance on manual labeling by 90% while maintaining competitive accuracy. In addition, the proposed model incorporates information from LiDAR and camera data through early fusion and leverages sparse convolution networks for real-time prediction. With its efficiency in both supervision and computation, we aim to extend MinkOcc beyond curated datasets, enabling broader real-world deployment of 3D semantic occupancy prediction in autonomous driving.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 45 of its 48 references that are in the map. First posted on arXiv 3 Apr 2025.

Builds on

45 of 48 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2023Segment Anythingcited by 8,234
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  5. 2017Squeeze-and-Excitation Networkscited by 4,789
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  7. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  8. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  9. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  10. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  11. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  12. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
33 more
  1. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  2. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  3. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
  4. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
  5. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  6. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 221
  7. 2017PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimationcited by 170
  8. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 165
  9. 2019MVX-Net: Multimodal VoxelNet for 3D Object Detectioncited by 163
  10. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 157
  11. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  12. 2022Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?cited by 128
  13. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 128
  14. 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 86
  15. 2020Generative Sparse Detection Networks for 3D Single-shot Object Detectioncited by 69
  16. 2023Fully Sparse 3D Occupancy Predictioncited by 64
  17. 2024FastOcc: Accelerating 3D Occupancy Prediction by Fusing the 2D Bird's-Eye View and Perspective Viewcited by 51
  18. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understandingcited by 46
  19. 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environmentscited by 41
  20. 2024A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspectivecited by 37
  21. 2024Sparse Voxels Rasterization: Real-time High-fidelity Radiance Field Renderingcited by 35
  22. 2024OccFusion: Multi-Sensor Fusion Framework for 3D Semantic Occupancy Predictioncited by 32
  23. 2024DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Predictioncited by 15
  24. 2024OccFusion: Depth Estimation Free Multi-sensor Fusion for 3D Occupancy Predictioncited by 13
  25. 2023Regulating Intermediate 3D Features for Vision-Centric Autonomous Drivingcited by 12
  26. 2024Real-time 3D semantic occupancy prediction for autonomous vehicles using memory-efficient sparse convolutioncited by 7
  27. 2024OccLoff: Learning Optimized Feature Fusion for 3D Occupancy Predictioncited by 7
  28. 2024Robust 3D Semantic Occupancy Prediction with Calibration-free Spatial Transformationcited by 7
  29. 2024GSRender: Deduplicated Occupancy Prediction via Weakly Supervised 3D Gaussian Splattingcited by 6
  30. 2024Semi-supervised 3D Semantic Scene Completion with 2D Vision Foundation Model Guidancecited by 5
  31. 20243D Representation Methods: A Surveycited by 5
  32. 2023SOccDPT: Semi-Supervised 3D Semantic Occupancy from Dense Prediction Transformers trained under memory constraintscited by 2
  33. 2024MR-Occ: Efficient Camera-LiDAR 3D Semantic Occupancy Prediction Using Hierarchical Multi-Resolution Voxel Representationcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Easy3D-Labels: Supervising Semantic Occupancy Estimation with 3D Pseudo-Labels for Automotive Perceptioncited by 2
  2. 2025QueryOcc: Query-based Self-Supervision for 3D Semantic Occupancycited by 2