# CVT-Occ: Cost Volume Temporal Fusion for 3D Occupancy Prediction

arXiv: https://arxiv.org/abs/2409.13430 · HTML: https://mlsanity.com/paper/2409.13430/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 40 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 9 of 9)

- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection](https://mlsanity.com/paper/2203.17054/) (2022)
- [Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving](https://mlsanity.com/paper/2304.14365/) (2023)
- [Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detection](https://mlsanity.com/paper/2210.02443/) (2022)
- [DPSNet: End-to-end Deep Plane Sweep Stereo](https://mlsanity.com/paper/1905.00538/) (2019)
- [OccDepth: A Depth-Aware Method for 3D Semantic Scene Completion](https://mlsanity.com/paper/2302.13540/) (2023)
- [SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Driving](https://mlsanity.com/paper/2306.09001/) (2023)
- [BEVStereo++: Accurate Depth Estimation in Multi-view 3D Object Detection via Dynamic Temporal Stereo](https://mlsanity.com/paper/2304.04185/) (2023)

## Built on by (most cited first, 24 of 24)

- [GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Prediction](https://mlsanity.com/paper/2412.10373/) (2024)
- [UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Driving](https://mlsanity.com/paper/2503.24381/) (2025)
- [DriveX: Omni Scene Modeling for Learning Generalizable World Knowledge in Autonomous Driving](https://mlsanity.com/paper/2505.19239/) (2025)
- [ALOcc: Adaptive Lifting-Based 3D Semantic Occupancy and Cost Volume-Based Flow Predictions](https://mlsanity.com/paper/2411.07725/) (2024)
- [GaussRender: Learning 3D Occupancy with Gaussian Rendering](https://mlsanity.com/paper/2502.05040/) (2025)
- [Language Driven Occupancy Prediction](https://mlsanity.com/paper/2411.16072/) (2024)
- [GS-Occ3D: Scaling Vision-only Occupancy Reconstruction with Gaussian Splatting](https://mlsanity.com/paper/2507.19451/) (2025)
- [SparseWorld: A Flexible, Adaptive, and Efficient 4D Occupancy World Model Powered by Sparse and Dynamic Queries](https://mlsanity.com/paper/2510.17482/) (2025)
- [LightOcc: Lightweight Spatial Embedding for Efficient Vision-based 3D Occupancy Prediction](https://mlsanity.com/paper/2412.05976/) (2024)
- [ShelfOcc: Native 3D Supervision beyond LiDAR for Vision-Based Occupancy Estimation](https://mlsanity.com/paper/2511.15396/) (2025)
- [OccAny: Generalized Unconstrained Urban 3D Occupancy](https://mlsanity.com/paper/2603.23502/) (2026)
- [One Step Closer: Creating the Future to Boost Monocular Semantic Scene Completion](https://mlsanity.com/paper/2507.13801/) (2025)
- [Out-of-Distribution Semantic Occupancy Prediction](https://mlsanity.com/paper/2506.21185/) (2025)
- [The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Models](https://mlsanity.com/paper/2601.08876/) (2026)
- [Collaborative Learning of Local 3D Occupancy Prediction and Versatile Global Occupancy Mapping](https://mlsanity.com/paper/2504.13596/) (2025)
- [S2GO: Streaming Sparse Gaussian Occupancy Prediction](https://mlsanity.com/paper/2506.05473/) (2025)
- [Humanoid-OmniOcc: Stereo-Based Full-View Occupancy Dataset for Embodied AI](https://mlsanity.com/paper/2606.22971/) (2026)
- [Deflickering Vision-Based Occupancy Networks through Lightweight Spatio-Temporal Correlation](https://mlsanity.com/paper/2502.15438/) (2025)
- [Towards Temporal Fusion Beyond the Field of View for Camera-based Semantic Scene Completion](https://mlsanity.com/paper/2511.12498/) (2025)
- [OccStress: Stress-Testing the 4D Occupancy Forecasting Chain](https://mlsanity.com/paper/2512.15621/) (2025)
- [StereoMV2D: A Sparse Temporal Stereo-Enhanced Framework for Robust Multi-View 3D Object Detection](https://mlsanity.com/paper/2512.17620/) (2025)
- [GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planning](https://mlsanity.com/paper/2605.17682/) (2026)
- [FDR-Occ: Factorized Dense Routing for Full-Spectrum 3D Occupancy Prediction](https://mlsanity.com/paper/2607.03822/) (2026)
- [SelectOccFlow: Selective Spatiotemporal Aggregation for 3D Occupancy and Scene Flow Prediction](https://mlsanity.com/paper/2610.04356/) (2026)

## Abstract (arXiv)

Vision-based 3D occupancy prediction is significantly challenged by the inherent limitations of monocular vision in depth estimation. This paper introduces CVT-Occ, a novel approach that leverages temporal fusion through the geometric correspondence of voxels over time to improve the accuracy of 3D occupancy predictions. By sampling points along the line of sight of each voxel and integrating the features of these points from historical frames, we construct a cost volume feature map that refines current volume features for improved prediction outcomes. Our method takes advantage of parallax cues from historical observations and employs a data-driven approach to learn the cost volume. We validate the effectiveness of CVT-Occ through rigorous experiments on the Occ3D-Waymo dataset, where it outperforms state-of-the-art methods in 3D occupancy prediction with minimal additional computational cost. The code is released at urlhttps://github.com/Tsinghua-MARS-Lab/CVT-Occ.