mlsanityData to 7 Oct 2026

Let Occ Flow: Self-Supervised 3D Occupancy Flow Prediction

arXiv 2407.07587 ↗First posted 10 Jul 2024cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

Accurate perception of the dynamic environment is a fundamental task for autonomous driving and robot systems. This paper introduces Let Occ Flow, the first self-supervised work for joint 3D occupancy and occupancy flow prediction using only camera inputs, eliminating the need for 3D annotations. Utilizing TPV for unified scene representation and deformable attention layers for feature aggregation, our approach incorporates a novel attention-based temporal fusion module to capture dynamic object dependencies, followed by a 3D refine module for fine-gained volumetric representation. Besides, our method extends differentiable rendering to 3D volumetric flow fields, leveraging zero-shot 2D segmentation and optical flow cues for dynamic decomposition and motion optimization. Extensive experiments on nuScenes and KITTI datasets demonstrate the competitive performance of our approach over prior state-of-the-art methods. Our project page is available at https://eliliu2233.github.io/letoccflow/

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 43 of its 48 references that are in the map. First posted on arXiv 10 Jul 2024.

Builds on

43 of 48 references.

References · 43

  1. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  3. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  4. 2022A ConvNet for the 2020scited by 3,545
  5. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  6. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  7. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  8. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  9. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  10. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  11. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  12. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
31 more
  1. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  2. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  3. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  4. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 553
  5. 2023CoTracker: It is Better to Track Togethercited by 516
  6. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  7. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  8. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  9. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  10. 2022Vision Transformer with Deformable Attentioncited by 293
  11. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  12. 2022Unifying Flow, Stereo and Depth Estimationcited by 270
  13. 2018FlowNet3D: Learning Scene Flow in 3D Point Cloudscited by 241
  14. 2023EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervisioncited by 216
  15. 2023Scene as Occupancycited by 208
  16. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 162
  17. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  18. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 126
  19. 2020What Matters in Unsupervised Optical Flowcited by 124
  20. 2024SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Predictioncited by 94
  21. 2023Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modelingcited by 92
  22. 2021Neural Scene Flow Priorcited by 91
  23. 2022SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimationcited by 76
  24. 2023Behind the Scenes: Density Fields for Single View Reconstructioncited by 63
  25. 2023Fully Sparse 3D Occupancy Predictioncited by 61
  26. 2022SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fieldscited by 49
  27. 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environmentscited by 39
  28. 2024ViewFormer: Exploring Spatiotemporal Modeling for Multi-View 3D Occupancy Perception via View-Guided Transformerscited by 36
  29. 2022SPIDR: SDF-based Neural Point Fields for Illumination and Deformationcited by 17
  30. 2024HENet: Hybrid Encoding for End-to-end Multi-task 3D Perception from Multi-view Camerascited by 16
  31. 2024nu-DBA: Neural Implicit Dense Bundle Adjustment Enables Image-Only Driving Scene Reconstructioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 14; 2026: 6.

23 papers.

Sort:

By area:

  1. 2024RenderWorld: World Model with Self-Supervised 3D Labelcited by 44
  2. 2025UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Drivingcited by 27
  3. 2025STCOcc: Sparse Spatial-Temporal Cascade Renovation for 3D Occupancy and Scene Flow Predictioncited by 20
  4. 2024ALOcc: Adaptive Lifting-Based 3D Semantic Occupancy and Cost Volume-Based Flow Predictionscited by 19
  5. 2024An Efficient Occupancy World Model via Decoupled Dynamic Flow and Image-assisted Trainingcited by 17
  6. 2025DrivePI: Spatial-aware 4D MLLM for Unified Autonomous Driving Understanding, Perception, Prediction and Planningcited by 15
  7. 2025GS-Occ3D: Scaling Vision-only Occupancy Reconstruction with Gaussian Splattingcited by 11
  8. 2025AGO: Adaptive Grounding for Open World 3D Occupancy Predictioncited by 9
  9. 2025Semantic Causality-Aware Vision-Based 3D Occupancy Predictioncited by 8
  10. 2025OneOcc: Semantic Occupancy Prediction for Legged Robots with a Single Panoramic Cameracited by 6
  11. 2025DIMO: Diverse 3D Motion Generation for Arbitrary Objectscited by 6
  12. 2025From 2D to 3D Cognition: A Brief Survey of General World Modelscited by 4
11 more
  1. 2025MetaOcc: Spatio-Temporal Fusion of Surround-View 4D Radar and Camera for 3D Occupancy Prediction with Dual Training Strategiescited by 3
  2. 2025ODG: Occupancy Prediction Using Dual Gaussianscited by 3
  3. 2025Easy3D-Labels: Supervising Semantic Occupancy Estimation with 3D Pseudo-Labels for Automotive Perceptioncited by 2
  4. 2025BePo: Dual Representation for 3D Occupancy Predictioncited by 1
  5. 2026Class-Distribution Guided Active Learning for 3D Occupancy Prediction in Autonomous Drivingcited by 1
  6. 2026InfraOcc: An Infrastructure Occupancy Benchmark with Static-to-Dynamic Reasoningcited by 1
  7. 2025GTAD: Global Temporal Aggregation Denoising Learning for 3D Semantic Occupancy Predictioncited by 0
  8. 2026SelfOccFlow: Towards end-to-end self-supervised 3D Occupancy Flow predictioncited by 0
  9. 2026LetOccVote: Learning Weakly Supervised 3D Occupancy through Consensuscited by 0
  10. 2026RoadOcc Learns When to Persist, Transport, or Refresh Memory for Roadside Occupancy Predictioncited by 0
  11. 2026TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perceptioncited by 0