# UAVScenes: A Multi-Modal Dataset for UAVs

arXiv: https://arxiv.org/abs/2507.22412 · HTML: https://mlsanity.com/paper/2507.22412/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 92 of its 125 references that are in the map.
- First posted on arXiv 30 Jul 2025.

## Builds on (most cited first, 30 of 92)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes](https://mlsanity.com/paper/2308.11417/) (2023)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [DeiT III: Revenge of the ViT](https://mlsanity.com/paper/2204.07118/) (2022)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation](https://mlsanity.com/paper/2404.15506/) (2024)
- [Point Transformer V2: Grouped Vector Attention and Partition-based Pooling](https://mlsanity.com/paper/2210.05666/) (2022)
- [Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image](https://mlsanity.com/paper/2307.10984/) (2023)
- [GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image](https://mlsanity.com/paper/2403.12013/) (2024)
- [The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking](https://mlsanity.com/paper/1804.00518/) (2018)
- [GaussianPro: 3D Gaussian Splatting with Progressive Propagation](https://mlsanity.com/paper/2402.14650/) (2024)
- [FloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understanding](https://mlsanity.com/paper/2012.02951/) (2020)

## Built on by (most cited first, 27 of 27)

- [iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Framework](https://mlsanity.com/paper/2605.03941/) (2026)
- [AirSim360: A Panoramic Simulation Platform within Drone View](https://mlsanity.com/paper/2512.02009/) (2025)
- [Is your VLM Sky-Ready? A Comprehensive Spatial Intelligence Benchmark for UAV Navigation](https://mlsanity.com/paper/2511.13269/) (2025)
- [Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligence](https://mlsanity.com/paper/2607.12477/) (2026)
- [Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systems](https://mlsanity.com/paper/2512.24385/) (2025)
- [Are VLMs Lost Between Sky and Space? LinkS2Bench for UAV-Satellite Dynamic Cross-View Spatial Intelligence](https://mlsanity.com/paper/2604.02020/) (2026)
- [AirZoo: A Unified Large-Scale Dataset for Grounding Aerial Geometric 3D Vision](https://mlsanity.com/paper/2604.26567/) (2026)
- [UAVFF3D: A Geometry-Aware Benchmark for Feed-Forward UAV 3D Reconstruction](https://mlsanity.com/paper/2605.17942/) (2026)
- [AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real World](https://mlsanity.com/paper/2606.29716/) (2026)
- [AeroDGS: Physically Consistent Dynamic Gaussian Splatting for Single-Sequence Aerial 4D Reconstruction](https://mlsanity.com/paper/2602.22376/) (2026)
- [SegFly: A Dataset and 2D-3D-2D Paradigm for Aerial RGB-Thermal Semantic Segmentation at Scale](https://mlsanity.com/paper/2603.17920/) (2026)
- [PiLoT: Neural Pixel-to-3D Registration for UAV-based Ego and Target Geo-localization](https://mlsanity.com/paper/2603.20778/) (2026)
- [OrthoTrack: Continuous 6-DoF UAV Trajectory Estimation Anchored in Public Orthophotos](https://mlsanity.com/paper/2606.25245/) (2026)
- [Advancing MLLM-based UAV Image Understanding and Reasoning: A Benchmark and a Training-Free Multi-Agent System](https://mlsanity.com/paper/2608.11738/) (2026)
- [GeoFF3D: Coordinate-Anchored Feed-Forward Reconstruction for Large-Scale UAV Mapping](https://mlsanity.com/paper/2608.28288/) (2026)
- [AdaptFly: Prompt-Guided Adaptation of Foundation Models for Low-Altitude UAV Networks](https://mlsanity.com/paper/2511.11720/) (2025)
- [Beyond a Single Light: A Large-Scale Aerial Dataset for Urban Scene Reconstruction Under Varying Illumination](https://mlsanity.com/paper/2512.14200/) (2025)
- [SOAR: Regression-based LiDAR Relocalization for UAVs](https://mlsanity.com/paper/2602.13267/) (2026)
- [PiLoT v2: Pixel-to-Orthogonal Map Alignment for Free-view UAV Geo-localization](https://mlsanity.com/paper/2606.31098/) (2026)
- [MulRobBench: A Decision-Level Benchmark for Safe and Security-Policy-Compliant Multimodal UAV Agents](https://mlsanity.com/paper/2607.23870/) (2026)
- [LAPF: LLM-Agent-Based Path Finder Using the UAVScenes Dataset](https://mlsanity.com/paper/2608.15175/) (2026)
- [VoxelFix: Post-Hoc Semantic Correction of Completed 3D Voxel Maps](https://mlsanity.com/paper/2609.05114/) (2026)
- [Aligned Radiometric RGB-Thermal Fusion for UAV Facade Anomaly Screening](https://mlsanity.com/paper/2609.12521/) (2026)
- [SkyAnchor: Updating Metric-scale Aerial 3D Gaussian Scenes from Unposed Ground-View Sequences](https://mlsanity.com/paper/2609.13903/) (2026)
- [PerSeM: Persistent Semantic Memory for Long-Horizon Open-Vocabulary UAV Mapping](https://mlsanity.com/paper/2609.19542/) (2026)
- [MultiFly: A Real-World Multimodal Aerial Dataset with Annotation-Efficient Label Transfer and Cross-Modal Semantic Consistency](https://mlsanity.com/paper/2610.10359/) (2026)
- [Can Synthetic UAV Benchmarks Be Trusted? Real-Synthetic Validation for UAV Model Selection](https://mlsanity.com/paper/2610.14647/) (2026)

## Abstract (arXiv)

Multi-modal perception is essential for unmanned aerial vehicle (UAV) operations, as it enables a comprehensive understanding of the UAVs' surrounding environment. However, most existing multi-modal UAV datasets are primarily biased toward localization and 3D reconstruction tasks, or only support map-level semantic segmentation due to the lack of frame-wise annotations for both camera images and LiDAR point clouds. This limitation prevents them from being used for high-level scene understanding tasks. To address this gap and advance multi-modal UAV perception, we introduce UAVScenes, a large-scale dataset designed to benchmark various tasks across both 2D and 3D modalities. Our benchmark dataset is built upon the well-calibrated multi-modal UAV dataset MARS-LVIG, originally developed only for simultaneous localization and mapping (SLAM). We enhance this dataset by providing manually labeled semantic annotations for both frame-wise images and LiDAR point clouds, along with accurate 6-degree-of-freedom (6-DoF) poses. These additions enable a wide range of UAV perception tasks, including segmentation, depth estimation, 6-DoF localization, place recognition, and novel view synthesis (NVS). Our dataset is available at https://github.com/sijieaaa/UAVScenes