# FALCON: Future-Aware Learning with Contextual Object-Centric Pretraining for UAV Action Recognition

arXiv: https://arxiv.org/abs/2409.18300 · HTML: https://mlsanity.com/paper/2409.18300/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 48 references that are in the map.
- First posted on arXiv 26 Sep 2024.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking](https://mlsanity.com/paper/2303.16727/) (2023)
- [Masked Autoencoders As Spatiotemporal Learners](https://mlsanity.com/paper/2205.09113/) (2022)
- [Spatiotemporal Contrastive Video Representation Learning](https://mlsanity.com/paper/2008.03800/) (2020)
- [Detection and Tracking Meet Drones Challenge](https://mlsanity.com/paper/2001.06303/) (2020)
- [Long-Term Feature Banks for Detailed Video Understanding](https://mlsanity.com/paper/1812.05038/) (2018)
- [Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers](https://mlsanity.com/paper/2106.05392/) (2021)
- [Videos as Space-Time Region Graphs](https://mlsanity.com/paper/1806.01810/) (2018)
- [A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning](https://mlsanity.com/paper/2104.14558/) (2021)
- [SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videos](https://mlsanity.com/paper/2206.07764/) (2022)
- [Towards Long-Form Video Understanding](https://mlsanity.com/paper/2106.11310/) (2021)
- [UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles](https://mlsanity.com/paper/2104.00946/) (2021)
- [Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learning](https://mlsanity.com/paper/2212.04500/) (2022)
- [Object-Region Video Transformers](https://mlsanity.com/paper/2110.06915/) (2021)
- [VIMPAC: Video Pre-Training via Masked Token Prediction and Contrastive Learning](https://mlsanity.com/paper/2106.11250/) (2021)
- [Effective Fusion Factor in FPN for Tiny Object Detection](https://mlsanity.com/paper/2011.02298/) (2020)
- [MAR: Masked Autoencoders for Efficient Action Recognition](https://mlsanity.com/paper/2207.11660/) (2022)
- [How can objects help action recognition?](https://mlsanity.com/paper/2306.11726/) (2023)
- [The State of Aerial Surveillance: A Survey](https://mlsanity.com/paper/2201.03080/) (2022)
- [FAR: Fourier Aerial Video Recognition](https://mlsanity.com/paper/2203.10694/) (2022)
- [Human Action Recognition in Drone Videos using a Few Aerial Training Examples](https://mlsanity.com/paper/1910.10027/) (2019)
- [AZTR: Aerial Video Action Recognition with Auto Zoom and Temporal Reasoning](https://mlsanity.com/paper/2303.01589/) (2023)

## Abstract (arXiv)

We introduce FALCON, a unified self-supervised video pretraining approach for UAV action recognition from raw RGB aerial footage, requiring no additional preprocessing at inference. UAV videos exhibit severe spatial imbalance: large, cluttered backgrounds dominate the field of view, causing reconstruction-based pretraining to waste capacity on uninformative regions and under-learn action-relevant human/object cues. FALCON addresses this by integrating object-aware masked autoencoding with object-centric dual-horizon future reconstruction. Using detections only during pretraining, we construct objectness priors that (i) enforce balanced token visibility during masking and (ii) concentrate reconstruction supervision on action-relevant regions, preventing learning from being dominated by background appearance. To promote temporal dynamics learning, we further reconstruct short- and long-horizon future content within an object-centric supervision region, injecting anticipatory temporal supervision that is robust to noisy aerial context. Across UAV benchmarks, FALCON improves top-1 accuracy by 2.9% on NEC-Drone and 5.8% on UAV-Human with a ViT-B backbone, while achieving 2×--5× faster inference than supervised approaches that rely on heavy test-time augmentation.