# OccuFly: A 3D Vision Benchmark for Semantic Scene Completion from the Aerial Perspective

arXiv: https://arxiv.org/abs/2512.20770 · HTML: https://mlsanity.com/paper/2512.20770/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 61 of its 90 references that are in the map.
- First posted on arXiv 23 Dec 2025.

## Builds on (most cited first, 30 of 61)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Depth Anything 3: Recovering the Visual Space from Any Views](https://mlsanity.com/paper/2511.10647/) (2025)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2302.07817/) (2023)
- [Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation](https://mlsanity.com/paper/2404.15506/) (2024)
- [MonoScene: Monocular 3D Semantic Scene Completion](https://mlsanity.com/paper/2112.00726/) (2021)
- [Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving](https://mlsanity.com/paper/2304.14365/) (2023)
- [SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving](https://mlsanity.com/paper/2303.09551/) (2023)
- [MapAnything: Universal Feed-Forward Metric 3D Reconstruction](https://mlsanity.com/paper/2509.13414/) (2025)
- [VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion](https://mlsanity.com/paper/2302.12251/) (2023)
- [OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication](https://mlsanity.com/paper/2109.07644/) (2021)
- [OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2304.05316/) (2023)
- [Detection and Tracking Meet Drones Challenge](https://mlsanity.com/paper/2001.06303/) (2020)
- [OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception](https://mlsanity.com/paper/2303.03991/) (2023)
- [MatrixCity: A Large-scale City Dataset for City-scale Neural Rendering and Beyond](https://mlsanity.com/paper/2309.16553/) (2023)
- [FloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understanding](https://mlsanity.com/paper/2012.02951/) (2020)
- [Capturing, Reconstructing, and Simulating: the UrbanScene3D Dataset](https://mlsanity.com/paper/2107.04286/) (2021)
- [Symphonize 3D Semantic Scene Completion with Contextual Instance Queries](https://mlsanity.com/paper/2306.15670/) (2023)
- [GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Prediction](https://mlsanity.com/paper/2412.04384/) (2024)
- [The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shot](https://mlsanity.com/paper/2306.16623/) (2023)
- [MobileStereoNet: Towards Lightweight Deep Networks for Stereo Matching](https://mlsanity.com/paper/2108.09770/) (2021)
- [SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Driving](https://mlsanity.com/paper/2306.09001/) (2023)
- [Context and Geometry Aware Voxel Transformer for Semantic Scene Completion](https://mlsanity.com/paper/2405.13675/) (2024)

## Built on by (most cited first, 5 of 5)

- [AirZoo: A Unified Large-Scale Dataset for Grounding Aerial Geometric 3D Vision](https://mlsanity.com/paper/2604.26567/) (2026)
- [AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real World](https://mlsanity.com/paper/2606.29716/) (2026)
- [EnerGS: Energy-Based Gaussian Splatting with Partial Geometric Priors](https://mlsanity.com/paper/2604.26238/) (2026)
- [SegFly: A Dataset and 2D-3D-2D Paradigm for Aerial RGB-Thermal Semantic Segmentation at Scale](https://mlsanity.com/paper/2603.17920/) (2026)
- [VoxelFix: Post-Hoc Semantic Correction of Completed 3D Voxel Maps](https://mlsanity.com/paper/2609.05114/) (2026)

## Abstract (arXiv)

Semantic Scene Completion (SSC) is essential for 3D perception in mobile robotics, as it enables holistic scene understanding by jointly estimating dense volumetric occupancy and per-voxel semantics. Although SSC has been widely studied in terrestrial domains such as autonomous driving, aerial settings like autonomous flying remain largely unexplored, thereby limiting progress on downstream applications. Furthermore, LiDAR sensors are the primary modality for SSC data generation, which poses challenges for most uncrewed aerial vehicles (UAVs) due to flight regulations, mass and energy constraints, and the sparsity of LiDAR point clouds from elevated viewpoints. To address these limitations, we propose a LiDAR-free, camera-based data generation framework. By leveraging classical 3D reconstruction, our framework automates semantic label transfer by lifting <10% of annotated images into the reconstructed point cloud, substantially minimizing manual 3D annotation effort. Based on this framework, we introduce OccuFly, the first real-world, camera-based aerial SSC benchmark, captured across multiple altitudes and all seasons. OccuFly provides over 20,000 samples of images, semantic voxel grids, and metric depth maps across 21 semantic classes in urban, industrial, and rural environments, and follows established data organization for seamless integration. We benchmark both SSC and metric monocular depth estimation on OccuFly, revealing fundamental limitations of current vision foundation models in aerial settings and establishing new challenges for robust 3D scene understanding in the aerial domain. Visit https://github.com/markus-42/occufly.