mlsanityData to 14 Oct 2026

ALOcc: Adaptive Lifting-Based 3D Semantic Occupancy and Cost Volume-Based Flow Predictions

arXiv 2411.07725 ↗First posted 12 Nov 2024cs.CVcited by 20

Not on the map. Why?

Abstract (arXiv)

3D semantic occupancy and flow prediction are fundamental to spatiotemporal scene understanding. This paper proposes a vision-based framework with three targeted improvements. First, we introduce an occlusion-aware adaptive lifting mechanism incorporating depth denoising. This enhances the robustness of 2D-to-3D feature transformation while mitigating reliance on depth priors. Second, we enforce 3D-2D semantic consistency via jointly optimized prototypes, using confidence- and category-aware sampling to address the long-tail classes problem. Third, to streamline joint prediction, we devise a BEV-centric cost volume to explicitly correlate semantic and flow features, supervised by a hybrid classification-regression scheme that handles diverse motion scales. Our purely convolutional architecture establishes new SOTA performance on multiple benchmarks for both semantic occupancy and joint occupancy semantic-flow prediction. We also present a family of models offering a spectrum of efficiency-performance trade-offs. Our real-time version exceeds all existing real-time methods in speed and accuracy, ensuring its practical viability.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 55 of its 70 references that are in the map. First posted on arXiv 12 Nov 2024.

Builds on

55 of 70 references.

References · 55

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  4. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  5. 2017Decoupled Weight Decay Regularizationcited by 1,831
  6. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  7. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  8. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  9. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  10. 2020AdaBins: Depth Estimation using Adaptive Binscited by 601
  11. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  12. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 488
38 more
  1. 2019PointRend: Image Segmentation as Renderingcited by 440
  2. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
  3. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
  4. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 341
  5. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  6. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
  7. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
  8. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 293
  9. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 221
  10. 2023Scene as Occupancycited by 209
  11. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 178
  12. 2022BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimationcited by 168
  13. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 165
  14. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 157
  15. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  16. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 128
  17. 20203D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Priorcited by 126
  18. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 115
  19. 2023FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugincited by 103
  20. 2023FB-BEV: BEV Representation from Forward-Backward View Transformationscited by 98
  21. 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 96
  22. 2023Fully Sparse 3D Occupancy Predictioncited by 64
  23. 2024OPUS: Occupancy Prediction Using a Sparse Setcited by 56
  24. 2019RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completioncited by 55
  25. 2023Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applicationscited by 52
  26. 2024FastOcc: Accelerating 3D Occupancy Prediction by Fusing the 2D Bird's-Eye View and Perspective Viewcited by 51
  27. 2024Efficient Deformable ConvNets: Rethinking Dynamic and Sparse Operator for Vision Applicationscited by 50
  28. 2024GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splattingcited by 50
  29. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  30. 2024Occupancy as Set of Pointscited by 38
  31. 2024ViewFormer: Exploring Spatiotemporal Modeling for Multi-View 3D Occupancy Perception via View-Guided Transformerscited by 37
  32. 2024Unleashing HyDRa: Hybrid Fusion, Depth Consistency and Radar for Unified 3D Perceptioncited by 35
  33. 2024OccFusion: Multi-Sensor Fusion Framework for 3D Semantic Occupancy Predictioncited by 32
  34. 2023UniOcc: Unifying Vision-Centric 3D Occupancy Prediction with Geometric and Semantic Renderingcited by 26
  35. 2024Let Occ Flow: Self-Supervised 3D Occupancy Flow Predictioncited by 24
  36. 2024CVT-Occ: Cost Volume Temporal Fusion for 3D Occupancy Predictioncited by 24
  37. 2023GeoMIM: Towards Better 3D Knowledge Transfer via Masked Image Modeling for Multi-view 3D Understandingcited by 17
  38. 2017Semantic 3D Occupancy Mapping through Efficient High Order CRFscited by 15

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 12.

20 papers.

Sort:

By area:

  1. 2025DrivePI: Spatial-aware 4D MLLM for Unified Autonomous Driving Understanding, Perception, Prediction and Planningcited by 15
  2. 2025Rethinking Temporal Fusion with a Unified Gradient Descent View for 3D Semantic Occupancy Predictioncited by 9
  3. 2025Semantic Causality-Aware Vision-Based 3D Occupancy Predictioncited by 8
  4. 2025OneOcc: Semantic Occupancy Prediction for Legged Robots with a Single Panoramic Cameracited by 8
  5. 2025ShelfOcc: Native 3D Supervision beyond LiDAR for Vision-Based Occupancy Estimationcited by 6
  6. 2026OccAny: Generalized Unconstrained Urban 3D Occupancycited by 6
  7. 2026SuperOcc: Toward Cohesive Temporal Modeling for Superquadric-based 3D Occupancy Predictioncited by 1
  8. 2026Gau-Occ: Geometry-Completed Gaussians for Multi-Modal 3D Occupancy Predictioncited by 1
  9. 2025Streaming Dense Voxel Representations for 3D Occupancy Predictioncited by 0
  10. 2025OccStress: Stress-Testing the 4D Occupancy Forecasting Chaincited by 0
  11. 2025HyGE-Occ: Hybrid View-Transformation with 3D Gaussian and Edge Priors for 3D Panoptic Occupancy Predictioncited by 0
  12. 2026SUG-Occ: Explicit Semantics and Uncertainty Guided Sparse Learning for Efficient 3D Occupancy Predictioncited by 0
8 more
  1. 2026ProOOD: Prototype-Guided Out-of-Distribution 3D Occupancy Predictioncited by 0
  2. 2026Height-Guided Projection Reparameterization for Camera-LiDAR Occupancycited by 0
  3. 2026GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planningcited by 0
  4. 2026FDR-Occ: Factorized Dense Routing for Full-Spectrum 3D Occupancy Predictioncited by 0
  5. 2026SparseOcc++: Geometry-Aware Sparse Latent Representation for Semantic Occupancy Predictioncited by 0
  6. 2026OPUS-V2: Bridging the Gap between Sparse Points and Dense Voxelscited by 0
  7. 2026RoadOcc Learns When to Persist, Transport, or Refresh Memory for Roadside Occupancy Predictioncited by 0
  8. 2026SelectOccFlow: Selective Spatiotemporal Aggregation for 3D Occupancy and Scene Flow Predictioncited by 0