# UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Driving

arXiv: https://arxiv.org/abs/2503.24381 · HTML: https://mlsanity.com/paper/2503.24381/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 92 references that are in the map.
- First posted on arXiv 31 Mar 2025.

## Builds on (most cited first, 30 of 59)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2302.07817/) (2023)
- [Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving](https://mlsanity.com/paper/2304.14365/) (2023)
- [SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving](https://mlsanity.com/paper/2303.09551/) (2023)
- [OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication](https://mlsanity.com/paper/2109.07644/) (2021)
- [OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving](https://mlsanity.com/paper/2311.16038/) (2023)
- [Human Motion Trajectory Prediction: A Survey](https://mlsanity.com/paper/1905.06113/) (2019)
- [CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers](https://mlsanity.com/paper/2207.02202/) (2022)
- [OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception](https://mlsanity.com/paper/2303.03991/) (2023)
- [Scene as Occupancy](https://mlsanity.com/paper/2306.02851/) (2023)
- [COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction](https://mlsanity.com/paper/2312.01919/) (2023)
- [OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Driving](https://mlsanity.com/paper/2412.15208/) (2024)
- [OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2409.03272/) (2024)
- [OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Driving](https://mlsanity.com/paper/2405.20337/) (2024)
- [Occupancy Flow Fields for Motion Forecasting in Autonomous Driving](https://mlsanity.com/paper/2203.03875/) (2022)
- [DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Model](https://mlsanity.com/paper/2410.10429/) (2024)
- [LOKI: Long Term and Key Intentions for Trajectory Prediction](https://mlsanity.com/paper/2108.08236/) (2021)
- [Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications](https://mlsanity.com/paper/2311.17663/) (2023)
- [UniTraj: A Unified Framework for Scalable Vehicle Trajectory Prediction](https://mlsanity.com/paper/2403.15098/) (2024)
- [STAMP: Scalable Task And Model-agnostic Collaborative Perception](https://mlsanity.com/paper/2501.18616/) (2025)
- [UnO: Unsupervised Occupancy Fields for Perception and Forecasting](https://mlsanity.com/paper/2406.08691/) (2024)
- [MotionSC: Data Set and Network for Real-Time Semantic Mapping in Dynamic Environments](https://mlsanity.com/paper/2203.07060/) (2022)
- [DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenes](https://mlsanity.com/paper/2410.18084/) (2024)

## Built on by (most cited first, 28 of 28)

- [3D and 4D World Modeling: A Survey](https://mlsanity.com/paper/2509.07996/) (2025)
- [OmniNWM: Omniscient Driving Navigation World Models](https://mlsanity.com/paper/2510.18313/) (2025)
- [SAFEFLOW: A Principled Protocol for Trustworthy and Transactional Autonomous Agent Systems](https://mlsanity.com/paper/2506.07564/) (2025)
- [GS-Occ3D: Scaling Vision-only Occupancy Reconstruction with Gaussian Splatting](https://mlsanity.com/paper/2507.19451/) (2025)
- [AGO: Adaptive Grounding for Open World 3D Occupancy Prediction](https://mlsanity.com/paper/2504.10117/) (2025)
- [ForecastOcc: Vision-based Semantic Occupancy Forecasting](https://mlsanity.com/paper/2602.08006/) (2026)
- [Towards Generalizable Safety in Crowd Navigation via Conformal Uncertainty Handling](https://mlsanity.com/paper/2508.05634/) (2025)
- [OccuFly: A 3D Vision Benchmark for Semantic Scene Completion from the Aerial Perspective](https://mlsanity.com/paper/2512.20770/) (2025)
- [Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Driving](https://mlsanity.com/paper/2603.25740/) (2026)
- [SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Model](https://mlsanity.com/paper/2511.22039/) (2025)
- [PaQ-DETR: Learning Pattern and Quality-Aware Dynamic Queries for Object Detection](https://mlsanity.com/paper/2603.06917/) (2026)
- [RDD: Retrieval-Based Demonstration Decomposer for Planner Alignment in Long-Horizon Tasks](https://mlsanity.com/paper/2510.14968/) (2025)
- [Dual-R-DETR: Resolving Query Competition with Pairwise Routing in Transformer Decoders](https://mlsanity.com/paper/2512.13876/) (2025)
- [SafeCoop: Unravelling Full Stack Safety in Agentic Collaborative Driving](https://mlsanity.com/paper/2510.18123/) (2025)
- [Priority-Aware Multi-Robot Coverage Path Planning](https://mlsanity.com/paper/2601.00580/) (2026)
- [DynFlowDrive: Flow-Based Dynamic World Modeling for Autonomous Driving](https://mlsanity.com/paper/2603.19675/) (2026)
- [InfraOcc: An Infrastructure Occupancy Benchmark with Static-to-Dynamic Reasoning](https://mlsanity.com/paper/2608.30657/) (2026)
- [Can we Trust Unreliable Voxels? Exploring 3D Semantic Occupancy Prediction under Label Noise](https://mlsanity.com/paper/2603.06279/) (2026)
- [M2-Occ: Resilient 3D Semantic Occupancy Prediction for Autonomous Driving with Incomplete Camera Inputs](https://mlsanity.com/paper/2603.09737/) (2026)
- [Dataset Safety in Autonomous Driving: Requirements, Risks, and Assurance](https://mlsanity.com/paper/2511.08439/) (2025)
- [NexusFlow: Unifying Disparate Tasks under Partial Supervision via Invertible Flow Networks](https://mlsanity.com/paper/2512.06251/) (2025)
- [OccStress: Stress-Testing the 4D Occupancy Forecasting Chain](https://mlsanity.com/paper/2512.15621/) (2025)
- [Rebenchmarking Unsupervised Monocular 3D Occupancy Prediction](https://mlsanity.com/paper/2602.06488/) (2026)
- [OccSim: Multi-kilometer Simulation with Long-horizon Occupancy World Models](https://mlsanity.com/paper/2603.28887/) (2026)
- [OccDirector: Language-Guided Behavior and Interaction Generation in 4D Occupancy Space](https://mlsanity.com/paper/2604.22240/) (2026)
- [Streaming Gaussian Encoding for 4D Panoptic Occupancy Tracking](https://mlsanity.com/paper/2606.30754/) (2026)
- [RoadOcc Learns When to Persist, Transport, or Refresh Memory for Roadside Occupancy Prediction](https://mlsanity.com/paper/2609.27677/) (2026)
- [RadarGSOcc: Doppler-Aware 3D Semantic Occupancy Prediction Using 4D Radar Tensor with Geometric-Semantic Feature Extraction](https://mlsanity.com/paper/2610.18686/) (2026)

## Abstract (arXiv)

We introduce UniOcc, a comprehensive, unified benchmark and toolkit for occupancy forecasting (i.e., predicting future occupancies based on historical information) and occupancy prediction (i.e., predicting current-frame occupancy from camera images. UniOcc unifies the data from multiple real-world datasets (i.e., nuScenes, Waymo) and high-fidelity driving simulators (i.e., CARLA, OpenCOOD), providing 2D/3D occupancy labels and annotating innovative per-voxel flows. Unlike existing studies that rely on suboptimal pseudo labels for evaluation, UniOcc incorporates novel evaluation metrics that do not depend on ground-truth labels, enabling robust assessment on additional aspects of occupancy quality. Through extensive experiments on state-of-the-art models, we demonstrate that large-scale, diverse training data and explicit flow information significantly enhance occupancy prediction and forecasting performance. Our data and code are available at https://uniocc.github.io/.