mlsanityData to 7 Oct 2026

STCOcc: Sparse Spatial-Temporal Cascade Renovation for 3D Occupancy and Scene Flow Prediction

arXiv 2504.19749 ↗First posted 28 Apr 2025cs.CVcited by 20

Not on the map. Why?

Abstract (arXiv)

3D occupancy and scene flow offer a detailed and dynamic representation of 3D scene. Recognizing the sparsity and complexity of 3D space, previous vision-centric methods have employed implicit learning-based approaches to model spatial and temporal information. However, these approaches struggle to capture local details and diminish the model's spatial discriminative ability. To address these challenges, we propose a novel explicit state-based modeling method designed to leverage the occupied state to renovate the 3D features. Specifically, we propose a sparse occlusion-aware attention mechanism, integrated with a cascade refinement strategy, which accurately renovates 3D features with the guidance of occupied state information. Additionally, we introduce a novel method for modeling long-term dynamic interactions, which reduces computational costs and preserves spatial information. Compared to the previous state-of-the-art methods, our efficient explicit renovation strategy not only delivers superior performance in terms of RayIoU and mAVE for occupancy and scene flow prediction but also markedly reduces GPU memory usage during training, bringing it down to 8.7GB. Our code is available on https://github.com/lzzzzzm/STCOcc

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 43 of its 54 references that are in the map. First posted on arXiv 28 Apr 2025.

Builds on

43 of 54 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  3. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  5. 2018SlowFast Networks for Video Recognitioncited by 1,862
  6. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  7. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  8. 2020Convolutional Occupancy Networkscited by 760
  9. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  10. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  11. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  12. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
31 more
  1. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  2. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  3. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  4. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  5. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  6. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
  7. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  8. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  9. 2023Scene as Occupancycited by 208
  10. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  11. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 177
  12. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 162
  13. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 161
  14. 2023SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videoscited by 149
  15. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 126
  16. 2023FB-BEV: BEV Representation from Forward-Backward View Transformationscited by 98
  17. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Drivingcited by 97
  18. 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 94
  19. 2024SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Predictioncited by 94
  20. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 88
  21. 2023OccDepth: A Depth-Aware Method for 3D Semantic Scene Completioncited by 85
  22. 2023Fully Sparse 3D Occupancy Predictioncited by 61
  23. 2024OPUS: Occupancy Prediction Using a Sparse Setcited by 56
  24. 2023Camera-based 3D Semantic Scene Completion with Sparse Guidance Networkcited by 40
  25. 2024Occupancy as Set of Pointscited by 37
  26. 2024ViewFormer: Exploring Spatiotemporal Modeling for Multi-View 3D Occupancy Perception via View-Guided Transformerscited by 36
  27. 2023Learning Occupancy for Monocular 3D Object Detectioncited by 35
  28. 2024Let Occ Flow: Self-Supervised 3D Occupancy Flow Predictioncited by 23
  29. 2023HeightFormer: Explicit Height Modeling without Extra Data for Camera-only 3D Object Detection in Bird's Eye Viewcited by 19
  30. 2023Regulating Intermediate 3D Features for Vision-Centric Autonomous Drivingcited by 12
  31. 2024WidthFormer: Toward Efficient Transformer-based BEV View Transformationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 12.

20 papers.

Sort:

By area:

  1. 2025I2-World: Intra-Inter Tokenization for Efficient Dynamic 4D Scene Forecastingcited by 8
  2. 2025ShelfOcc: Native 3D Supervision beyond LiDAR for Vision-Based Occupancy Estimationcited by 6
  3. 2026ForecastOcc: Vision-based Semantic Occupancy Forecastingcited by 6
  4. 2025SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Modelcited by 3
  5. 2025See through the Dark: Learning Illumination-affined Representations for Nighttime Occupancy Predictioncited by 2
  6. 2025Easy3D-Labels: Supervising Semantic Occupancy Estimation with 3D Pseudo-Labels for Automotive Perceptioncited by 2
  7. 2026The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Modelscited by 2
  8. 2026Gau-Occ: Geometry-Completed Gaussians for Multi-Modal 3D Occupancy Predictioncited by 1
  9. 2026InfraOcc: An Infrastructure Occupancy Benchmark with Static-to-Dynamic Reasoningcited by 1
  10. 2025FMOcc: TPV-Driven Flow Matching for 3D Occupancy Prediction with Selective State Space Modelcited by 0
  11. 2025SuperQuadricOcc: Real-Time Self-Supervised Semantic Occupancy Estimation with Superquadric Volume Renderingcited by 0
  12. 2025OccStress: Stress-Testing the 4D Occupancy Forecasting Chaincited by 0
8 more
  1. 2026SuperOcc: Toward Cohesive Temporal Modeling for Superquadric-based 3D Occupancy Predictioncited by 0
  2. 2026FreeOcc: Training-free Panoptic Occupancy Prediction via Foundation Modelscited by 0
  3. 2026M2-Occ: Resilient 3D Semantic Occupancy Prediction for Autonomous Driving with Incomplete Camera Inputscited by 0
  4. 2026Height-Guided Projection Reparameterization for Camera-LiDAR Occupancycited by 0
  5. 2026GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planningcited by 0
  6. 2026Geometry-Aware Spatio-Temporal Context Modeling for 4D Occupancy Forecastingcited by 0
  7. 2026OPUS-V2: Bridging the Gap between Sparse Points and Dense Voxelscited by 0
  8. 2026RoadOcc Learns When to Persist, Transport, or Refresh Memory for Roadside Occupancy Predictioncited by 0