# UAV-MM3D: A Large-Scale Synthetic Benchmark for 3D Perception of Unmanned Aerial Vehicles with Multi-Modal Data

arXiv: https://arxiv.org/abs/2511.22404 · HTML: https://mlsanity.com/paper/2511.22404/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 27 Nov 2025.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework](https://mlsanity.com/paper/2203.11991/) (2022)
- [DexYCB: A Benchmark for Capturing Hand Grasping of Objects](https://mlsanity.com/paper/2104.04631/) (2021)
- [LLVIP: A Visible-infrared Paired Dataset for Low-light Vision](https://mlsanity.com/paper/2108.10831/) (2021)
- [One Million Scenes for Autonomous Driving: ONCE Dataset](https://mlsanity.com/paper/2106.11037/) (2021)
- [T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objects](https://mlsanity.com/paper/1701.05498/) (2017)
- [Real-Time Seamless Single Shot 6D Object Pose Prediction](https://mlsanity.com/paper/1711.08848/) (2017)
- [Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotations](https://mlsanity.com/paper/2012.09988/) (2020)
- [ODTrack: Online Dense Temporal Token Learning for Visual Tracking](https://mlsanity.com/paper/2401.01686/) (2024)
- [Vision-based Anti-UAV Detection and Tracking](https://mlsanity.com/paper/2205.10851/) (2022)
- [Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications](https://mlsanity.com/paper/1708.05869/) (2017)
- [MMAUD: A Comprehensive Multi-Modal Anti-UAV Dataset for Modern Miniature Drone Threats](https://mlsanity.com/paper/2402.03706/) (2024)
- [Evidential Detection and Tracking Collaboration: New Problem, Benchmark and Algorithm for Robust Anti-UAV System](https://mlsanity.com/paper/2306.15767/) (2023)
- [DrIFT: Autonomous Drone Dataset with Integrated Real and Synthetic Data, Flexible Views, and Transformed Domains](https://mlsanity.com/paper/2412.04789/) (2024)
- [Domain Adaptive Detection of MAVs: A Benchmark and Noise Suppression Network](https://mlsanity.com/paper/2403.16669/) (2024)

## Built on by (most cited first, 3 of 3)

- [PRISM: Color-Stratified Point Cloud Sampling](https://mlsanity.com/paper/2601.06839/) (2026)
- [Residual Kalman Dynamics for Event-Based UAV Forecasting](https://mlsanity.com/paper/2609.00839/) (2026)
- [MultiFly: A Real-World Multimodal Aerial Dataset with Annotation-Efficient Label Transfer and Cross-Modal Semantic Consistency](https://mlsanity.com/paper/2610.10359/) (2026)

## Abstract (arXiv)

Accurate perception of UAVs in complex low-altitude environments is critical for airspace security and related intelligent systems. Developing reliable solutions requires large-scale, accurately annotated, and multimodal data. However, real-world UAV data collection faces inherent constraints due to airspace regulations, privacy concerns, and environmental variability, while manual annotation of 3D poses and cross-modal correspondences is time-consuming and costly. To overcome these challenges, we introduce UAV-MM3D, a high-fidelity multimodal synthetic dataset for low-altitude UAV perception and motion understanding. It comprises 400K synchronized frames across diverse scenes (urban areas, suburbs, forests, coastal regions) and weather conditions (clear, cloudy, rainy, foggy), featuring multiple UAV models (micro, small, medium-sized) and five modalities - RGB, IR, LiDAR, Radar, and DVS (Dynamic Vision Sensor). Each frame provides 2D/3D bounding boxes, 6-DoF poses, and instance-level annotations, enabling core tasks related to UAVs such as 3D detection, pose estimation, target tracking, and short-term trajectory forecasting. We further propose LGFusionNet, a LiDAR-guided multimodal fusion baseline, and a dedicated UAV trajectory prediction baseline to facilitate benchmarking. With its controllable simulation environment, comprehensive scenario coverage, and rich annotations, UAV3D offers a public benchmark for advancing 3D perception of UAVs.