mlsanityData to 14 Oct 2026

LiDAR-based 4D Occupancy Completion and Forecasting

arXiv 2310.11239 ↗First posted 17 Oct 2023cs.CV cs.ROcited by 23

Not on the map. Why?

Abstract (arXiv)

Scene completion and forecasting are two popular perception problems in research for mobile agents like autonomous vehicles. Existing approaches treat the two problems in isolation, resulting in a separate perception of the two aspects. In this paper, we introduce a novel LiDAR perception task of Occupancy Completion and Forecasting (OCF) in the context of autonomous driving to unify these aspects into a cohesive framework. This task requires new algorithms to address three challenges altogether: (1) sparse-to-dense reconstruction, (2) partial-to-complete hallucination, and (3) 3D-to-4D prediction. To enable supervision and evaluation, we curate a large-scale dataset termed OCFBench from public autonomous driving datasets. We analyze the performance of closely related existing baseline models and our own ones on our dataset. We envision that this research will inspire and call for further investigation in this evolving and crucial area of 4D perception. Our code for data curation and baseline implementation is available at https://github.com/ai4ce/Occ4cast.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 52 of its 62 references that are in the map. First posted on arXiv 17 Oct 2023.

Builds on

52 of 62 references.

References · 52

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  4. 2021Vision Transformers for Dense Predictioncited by 1,803
  5. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  6. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  7. 2022Planning-oriented Autonomous Drivingcited by 1,087
  8. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
  9. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  10. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
  11. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  12. 2016Semantic Scene Completion from a Single Depth Imagecited by 472
38 more
  1. 2020Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detectioncited by 410
  2. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
  3. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
  4. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  5. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
  6. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
  7. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 295
  8. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 293
  9. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  10. 2021End-to-end Interpretable Neural Motion Plannercited by 242
  11. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 240
  12. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  13. 2021One Million Scenes for Autonomous Driving: ONCE Datasetcited by 218
  14. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 212
  15. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 194
  16. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  17. 2020A2D2: Audi Autonomous Driving Datasetcited by 190
  18. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 188
  19. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 153
  20. 2020S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Cloudscited by 102
  21. 2021TPCN: Temporal Point Cloud Networks for Motion Forecastingcited by 94
  22. 2022Differentiable Raycasting for Self-supervised Occupancy Forecastingcited by 84
  23. 2023SCPNet: Semantic Scene Completion on Point Cloudcited by 80
  24. 2023Point Cloud Forecasting as a Proxy for 4D Occupancy Forecastingcited by 78
  25. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 75
  26. 2019A*3D Dataset: Towards Autonomous Driving in Challenging Environmentscited by 65
  27. 2022Occupancy Flow Fields for Motion Forecasting in Autonomous Drivingcited by 62
  28. 2022Comprehensive Review of Deep Learning-Based 3D Point Cloud Completion Processing and Analysiscited by 44
  29. 2021Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networkscited by 42
  30. 2018Long-Term Occupancy Grid Prediction Using Recurrent Neural Networkscited by 29
  31. 2017Im2Pano3D: Extrapolating 360 Structure and Semantics Beyond the Field of Viewcited by 25
  32. 2023Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Reviewcited by 25
  33. 2018Extreme Relative Pose Estimation for RGB-D Scans via Scene Completioncited by 22
  34. 2021Heterogeneous-Agent Trajectory Forecasting Incorporating Class Uncertaintycited by 19
  35. 2019CloudLSTM: A Recurrent Neural Model for Spatiotemporal Point-cloud Stream Forecastingcited by 14
  36. 2020Point Cloud Completion by Learning Shape Priorscited by 14
  37. 2022DeepMapping2: Self-Supervised Large-Scale LiDAR Map Optimizationcited by 14
  38. 2020Volumetric Occupancy Mapping With Probabilistic Depth Completion for Robotic Navigationcited by 13

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 9; 2025: 10; 2026: 3.

23 papers.

Sort:

By area:

  1. 2024DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Drivingcited by 93
  2. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 78
  3. 2025A Survey of World Models for Autonomous Drivingcited by 60
  4. 2024A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspectivecited by 37
  5. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  6. 2024RadarOcc: Robust 3D Occupancy Prediction with 4D Imaging Radarcited by 33
  7. 2024OccFusion: Multi-Sensor Fusion Framework for 3D Semantic Occupancy Predictioncited by 32
  8. 2025GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Controlcited by 19
  9. 2025GaussianFormer3D: Multi-Modal Gaussian-based Semantic Occupancy Prediction with 3D Deformable Attentioncited by 14
  10. 2025COME: Adding Scene-Centric Forecasting Control to Occupancy World Modelcited by 14
  11. 2024HybridOcc: NeRF Enhanced Transformer-based Multi-Camera 3D Occupancy Predictioncited by 13
  12. 2025GS-Occ3D: Scaling Vision-only Occupancy Reconstruction with Gaussian Splattingcited by 11
11 more
  1. 2025Towards foundational LiDAR world models with efficient latent flow matchingcited by 10
  2. 2024Joint Perception and Prediction for Autonomous Driving: A Surveycited by 7
  3. 2025Beyond One Shot, Beyond One Perspective: Cross-View and Long-Horizon Distillation for Better LiDAR Representationscited by 7
  4. 2024Spatiotemporal Decoupling for Efficient Vision-Based Occupancy Forecastingcited by 6
  5. 2025OccuFly: A 3D Vision Benchmark for Semantic Scene Completion from the Aerial Perspectivecited by 5
  6. 2026Self-Supervised JEPA-based World Models for LiDAR Occupancy Completion and Forecastingcited by 5
  7. 2024A Comprehensive Review of 3D Object Detection in Autonomous Driving: Technological Advances and Future Directionscited by 2
  8. 2025To New Beginnings: A Survey of Unified Perception in Autonomous Vehicle Softwarecited by 1
  9. 2025SliceSemOcc: Vertical Slice Based Multimodal 3D Semantic Occupancy Representationcited by 1
  10. 2026Visual Implicit Geometry Transformer for Autonomous Drivingcited by 0
  11. 2026Towards Collaborative Joint Perception and Prediction: Framework, Baseline Evaluation, and Deployment Perspectivescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024UnO: Unsupervised Occupancy Fields for Perception and Forecasting
  2. 2023Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications
  3. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving
  4. 2024Self-supervised Multi-future Occupancy Forecasting for Autonomous Driving
  5. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving
  6. 2025OccProphet: Pushing Efficiency Frontier of Camera-Only 4D Occupancy Forecasting with Observer-Forecaster-Refiner Framework
  7. 2025Vision-Centric 4D Occupancy Forecasting and Planning via Implicit Residual World Models
  8. 2025OccStress: Stress-Testing the 4D Occupancy Forecasting Chain