# UAV3D: A Large-scale 3D Perception Benchmark for Unmanned Aerial Vehicles

arXiv: https://arxiv.org/abs/2410.11125 · HTML: https://mlsanity.com/paper/2410.11125/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 35 references that are in the map.
- First posted on arXiv 14 Oct 2024.

## Builds on (most cited first, 30 of 31)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [PETR: Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2203.05625/) (2022)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images](https://mlsanity.com/paper/2206.01256/) (2022)
- [V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer](https://mlsanity.com/paper/2203.10638/) (2022)
- [Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection](https://mlsanity.com/paper/2303.11926/) (2023)
- [OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication](https://mlsanity.com/paper/2109.07644/) (2021)
- [V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction](https://mlsanity.com/paper/2008.07519/) (2020)
- [DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection](https://mlsanity.com/paper/2204.05575/) (2022)
- [Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps](https://mlsanity.com/paper/2209.12836/) (2022)
- [The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking](https://mlsanity.com/paper/1804.00518/) (2018)
- [Learning Distilled Collaboration Graph for Multi-Agent Perception](https://mlsanity.com/paper/2111.00643/) (2021)
- [When2com: Multi-Agent Perception via Communication Graph Grouping](https://mlsanity.com/paper/2006.00176/) (2020)
- [V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving](https://mlsanity.com/paper/2202.08449/) (2022)
- [V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception](https://mlsanity.com/paper/2303.07601/) (2023)
- [SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videos](https://mlsanity.com/paper/2308.09244/) (2023)
- [Who2com: Collaborative Perception via Learnable Handshake Communication](https://mlsanity.com/paper/2003.09575/) (2020)
- [Sparse4D v2: Recurrent Temporal Fusion with Sparse Model](https://mlsanity.com/paper/2305.14018/) (2023)
- [Vision Meets Drones: A Challenge](https://mlsanity.com/paper/1804.07437/) (2018)
- [TUMTraf V2X Cooperative Perception Dataset](https://mlsanity.com/paper/2403.01316/) (2024)
- [V2X-Real: a Large-Scale Dataset for Vehicle-to-Everything Cooperative Perception](https://mlsanity.com/paper/2403.16034/) (2024)
- [RCooper: A Real-world Large-scale Dataset for Roadside Cooperative Perception](https://mlsanity.com/paper/2403.10145/) (2024)
- [BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection](https://mlsanity.com/paper/2312.01696/) (2023)
- [S2R-ViT for Multi-Agent Cooperative Perception: Bridging the Gap from Simulation to Reality](https://mlsanity.com/paper/2307.07935/) (2023)

## Built on by (most cited first, 16 of 16)

- [Griffin: Aerial-Ground Cooperative Detection and Tracking Dataset and Benchmark](https://mlsanity.com/paper/2503.06983/) (2025)
- [AirV2X: Unified Air-Ground Vehicle-to-Everything Collaboration](https://mlsanity.com/paper/2506.19283/) (2025)
- [α3-Bench: A Unified Benchmark of Safety, Robustness, and Efficiency for LLM-Based UAV Agents over 6G Networks](https://mlsanity.com/paper/2601.03281/) (2026)
- [OccuFly: A 3D Vision Benchmark for Semantic Scene Completion from the Aerial Perspective](https://mlsanity.com/paper/2512.20770/) (2025)
- [Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systems](https://mlsanity.com/paper/2512.24385/) (2025)
- [UAV-VL-R1: Generalizing Vision-Language Models via Supervised Fine-Tuning and Multi-Stage GRPO for UAV Visual Reasoning](https://mlsanity.com/paper/2508.11196/) (2025)
- [Embodied4C: Measuring What Matters for Embodied Vision-Language Navigation](https://mlsanity.com/paper/2512.18028/) (2025)
- [V2U4Real: A Real-world Large-scale Dataset for Vehicle-to-UAV Cooperative Perception](https://mlsanity.com/paper/2603.25275/) (2026)
- [Is Intermediate Fusion All You Need for UAV-based Collaborative Perception?](https://mlsanity.com/paper/2504.21774/) (2025)
- [Probabilistic Hyper-Graphs using Multiple Randomly Masked Autoencoders for Semi-supervised Multi-modal Multi-task Learning](https://mlsanity.com/paper/2510.10068/) (2025)
- [AeroDGS: Physically Consistent Dynamic Gaussian Splatting for Single-Sequence Aerial 4D Reconstruction](https://mlsanity.com/paper/2602.22376/) (2026)
- [AdaptFly: Prompt-Guided Adaptation of Foundation Models for Low-Altitude UAV Networks](https://mlsanity.com/paper/2511.11720/) (2025)
- [SkyShield: Occupancy as a Safety Interface for Low-Altitude UAV Autonomy](https://mlsanity.com/paper/2606.00747/) (2026)
- [CAMF-Det: Closure-Aware Multimodal Fusion for LiDAR-Camera 3D Object Detection on UAV Platforms](https://mlsanity.com/paper/2606.09143/) (2026)
- [HydraCollab: Adaptive Collaborative-Perception for Distributed Autonomous Systems](https://mlsanity.com/paper/2607.00191/) (2026)
- [CERF: Communication-Efficient and Retraining-Free Collaborative Perception](https://mlsanity.com/paper/2609.00951/) (2026)

## Abstract (arXiv)

Unmanned Aerial Vehicles (UAVs), equipped with cameras, are employed in numerous applications, including aerial photography, surveillance, and agriculture. In these applications, robust object detection and tracking are essential for the effective deployment of UAVs. However, existing benchmarks for UAV applications are mainly designed for traditional 2D perception tasks, restricting the development of real-world applications that require a 3D understanding of the environment. Furthermore, despite recent advancements in single-UAV perception, limited views of a single UAV platform significantly constrain its perception capabilities over long distances or in occluded areas. To address these challenges, we introduce UAV3D, a benchmark designed to advance research in both 3D and collaborative 3D perception tasks with UAVs. UAV3D comprises 1,000 scenes, each of which has 20 frames with fully annotated 3D bounding boxes on vehicles. We provide the benchmark for four 3D perception tasks: single-UAV 3D object detection, single-UAV object tracking, collaborative-UAV 3D object detection, and collaborative-UAV object tracking. Our dataset and code are available at https://huiyegit.github.io/UAV3D_Benchmark/.