# UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles

arXiv: https://arxiv.org/abs/2104.00946 · HTML: https://mlsanity.com/paper/2104.00946/

## Key facts

- Cited by 96 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 41 references that are in the map.
- First posted on arXiv 2 Apr 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline)](https://mlsanity.com/paper/1711.09349/) (2017)
- [Bag of Tricks and A Strong Baseline for Deep Person Re-identification](https://mlsanity.com/paper/1903.07071/) (2019)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [Temporal Segment Networks for Action Recognition in Videos](https://mlsanity.com/paper/1705.02953/) (2017)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [Human Action Recognition and Prediction: A Survey](https://mlsanity.com/paper/1806.11230/) (2018)
- [Joint Discriminative and Generative Learning for Person Re-identification](https://mlsanity.com/paper/1904.07223/) (2019)
- [PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action Understanding](https://mlsanity.com/paper/1703.07475/) (2017)
- [Part-Aligned Bilinear Representations for Person Re-identification](https://mlsanity.com/paper/1804.07094/) (2018)
- [Okutama-Action: An Aerial View Video Dataset for Concurrent Human Action Detection](https://mlsanity.com/paper/1706.03038/) (2017)
- [Person Re-identification in Aerial Imagery](https://mlsanity.com/paper/1908.05024/) (2019)
- [Histogram of Oriented Principal Components for Cross-View Action Recognition](https://mlsanity.com/paper/1409.6813/) (2014)
- [Eye in the Sky: Real-time Drone Surveillance System (DSS) for Violent Individuals Identification using ScatterNet Hybrid Deep Learning Network](https://mlsanity.com/paper/1806.00746/) (2018)
- [UAV-GESTURE: A Dataset for UAV Control and Gesture Recognition](https://mlsanity.com/paper/1901.02602/) (2019)
- [A Large-scale Varying-view RGB-D Action Dataset for Arbitrary-view Human Action Recognition](https://mlsanity.com/paper/1904.10681/) (2019)
- [EV-Action: Electromyography-Vision Multi-Modal Action Dataset](https://mlsanity.com/paper/1904.12602/) (2019)

## Built on by (most cited first, 30 of 96)

- [Efficient Video Transformers with Spatial-Temporal Token Selection](https://mlsanity.com/paper/2111.11591/) (2021)
- [UniHCP: A Unified Model for Human-Centric Perceptions](https://mlsanity.com/paper/2303.02936/) (2023)
- [Skeleton Cloud Colorization for Unsupervised 3D Action Representation Learning](https://mlsanity.com/paper/2108.01959/) (2021)
- [UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobility](https://mlsanity.com/paper/2501.02341/) (2025)
- [MAtch, eXpand and Improve: Unsupervised Finetuning for Zero-Shot Action Recognition with Language Knowledge](https://mlsanity.com/paper/2303.08914/) (2023)
- [HumanBench: Towards General Human-centric Perception with Projector Assisted Pretraining](https://mlsanity.com/paper/2303.05675/) (2023)
- [LLMs are Good Action Recognizers](https://mlsanity.com/paper/2404.00532/) (2024)
- [UNIK: A Unified Framework for Real-world Skeleton-based Action Recognition](https://mlsanity.com/paper/2107.08580/) (2021)
- [WebUAV-3M: A Benchmark for Unveiling the Power of Million-Scale Deep UAV Tracking](https://mlsanity.com/paper/2201.07425/) (2022)
- [Towards a Unified View on Visual Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2210.00788/) (2022)
- [View-decoupled Transformer for Person Re-identification under Aerial-ground Camera Network](https://mlsanity.com/paper/2403.14513/) (2024)
- [ViA: View-invariant Skeleton Action Representation Learning via Motion Retargeting](https://mlsanity.com/paper/2209.00065/) (2022)
- [Transformer for Object Re-Identification: A Survey](https://mlsanity.com/paper/2401.06960/) (2024)
- [AG-ReID.v2: Bridging Aerial and Ground Views for Person Re-identification](https://mlsanity.com/paper/2401.02634/) (2024)
- [Aerial-Ground Person Re-ID](https://mlsanity.com/paper/2303.08597/) (2023)
- [Exploring Incompatible Knowledge Transfer in Few-shot Image Generation](https://mlsanity.com/paper/2304.07574/) (2023)
- [Transformers in Healthcare: A Survey](https://mlsanity.com/paper/2307.00067/) (2023)
- [Synthetic-to-Real Domain Adaptation for Action Recognition: A Dataset and Baseline Performances](https://mlsanity.com/paper/2303.10280/) (2023)
- [SeCap: Self-Calibrating and Adaptive Prompts for Cross-view Person Re-Identification in Aerial-Ground Networks](https://mlsanity.com/paper/2503.06965/) (2025)
- [Frequency Guidance Matters: Skeletal Action Recognition by Frequency-Aware Mixed Transformer](https://mlsanity.com/paper/2407.12322/) (2024)
- [RingMo-Aerial: An Aerial Remote Sensing Foundation Model With Affine Transformation Contrastive Learning](https://mlsanity.com/paper/2409.13366/) (2024)
- [A Large-scale Study of Spatiotemporal Representation Learning with a New Benchmark on Action Recognition](https://mlsanity.com/paper/2303.13505/) (2023)
- [MotionBank: A Large-scale Video Motion Benchmark with Disentangled Rule-based Annotations](https://mlsanity.com/paper/2410.13790/) (2024)
- [Skeleton-Based Mutually Assisted Interacted Object Localization and Human Action Recognition](https://mlsanity.com/paper/2110.14994/) (2021)
- [SequencePAR: Understanding Pedestrian Attributes via A Sequence Generation Paradigm](https://mlsanity.com/paper/2312.01640/) (2023)
- [UAVs Meet Agentic AI: A Multidomain Survey of Autonomous Aerial Intelligence and Agentic UAVs](https://mlsanity.com/paper/2506.08045/) (2025)
- [Skeleton-based Action Recognition via Adaptive Cross-Form Learning](https://mlsanity.com/paper/2206.15085/) (2022)
- [Self-Supervised Video Representation Learning via Latent Time Navigation](https://mlsanity.com/paper/2305.06437/) (2023)
- [Rotation Invariant Transformer for Recognizing Object in UAVs](https://mlsanity.com/paper/2311.02559/) (2023)
- [Foundation Model for Skeleton-Based Human Action Understanding](https://mlsanity.com/paper/2508.12586/) (2025)

## Abstract (arXiv)

Human behavior understanding with unmanned aerial vehicles (UAVs) is of great significance for a wide range of applications, which simultaneously brings an urgent demand of large, challenging, and comprehensive benchmarks for the development and evaluation of UAV-based models. However, existing benchmarks have limitations in terms of the amount of captured data, types of data modalities, categories of provided tasks, and diversities of subjects and environments. Here we propose a new benchmark - UAVHuman - for human behavior understanding with UAVs, which contains 67,428 multi-modal video sequences and 119 subjects for action recognition, 22,476 frames for pose estimation, 41,290 frames and 1,144 identities for person re-identification, and 22,263 frames for attribute recognition. Our dataset was collected by a flying UAV in multiple urban and rural districts in both daytime and nighttime over three months, hence covering extensive diversities w.r.t subjects, backgrounds, illuminations, weathers, occlusions, camera motions, and UAV flying attitudes. Such a comprehensive and challenging benchmark shall be able to promote the research of UAV-based human behavior understanding, including action recognition, pose estimation, re-identification, and attribute recognition. Furthermore, we propose a fisheye-based action recognition method that mitigates the distortions in fisheye videos via learning unbounded transformations guided by flat RGB videos. Experiments show the efficacy of our method on the UAV-Human dataset. The project page: https://github.com/SUTDCV/UAV-Human