# CognitiveDrone: A VLA Model and Evaluation Benchmark for Real-Time Cognitive Task Solving and Reasoning in UAVs

arXiv: https://arxiv.org/abs/2503.01378 · HTML: https://mlsanity.com/paper/2503.01378/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 19 of its 27 references that are in the map.
- First posted on arXiv 3 Mar 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)
- Vision-language-action models / [Aerial and drone VLA](https://mlsanity.com/vla/aerial-and-drone-vla/)

## Builds on (most cited first, 19 of 19)

- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [Open X-Embodiment: Robotic Learning Datasets and RT-X Models](https://mlsanity.com/paper/2310.08864/) (2023)
- [ChatGPT for Robotics: Design Principles and Model Abilities](https://mlsanity.com/paper/2306.17582/) (2023)
- [VLP: Vision Language Planning for Autonomous Driving](https://mlsanity.com/paper/2401.05577/) (2024)
- [AerialVLN: Vision-and-Language Navigation for UAVs](https://mlsanity.com/paper/2308.06735/) (2023)
- [Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology](https://mlsanity.com/paper/2410.07087/) (2024)
- [PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks](https://mlsanity.com/paper/2411.00081/) (2024)
- [Aerial Vision-and-Dialog Navigation](https://mlsanity.com/paper/2205.12219/) (2022)
- [UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobility](https://mlsanity.com/paper/2501.02341/) (2025)
- [UAV-VLA: Vision-Language-Action System for Large Scale Aerial Mission Generation](https://mlsanity.com/paper/2501.05014/) (2025)
- [Exploring Spatial Representation to Enhance LLM Reasoning in Aerial Vision-Language Navigation](https://mlsanity.com/paper/2410.08500/) (2024)
- [Swarm-GPT: Combining Large Language Models with Safe Motion Planning for Robot Choreography Design](https://mlsanity.com/paper/2312.01059/) (2023)
- [A Safer Vision-based Autonomous Planning System for Quadrotor UAVs with Dynamic Obstacle Trajectory Prediction and Its Application with LLMs](https://mlsanity.com/paper/2311.12893/) (2023)
- [Helpful DoggyBot: Open-World Object Fetching using Legged Robots and Vision-Language Models](https://mlsanity.com/paper/2410.00231/) (2024)
- [CognitiveOS: Large Multimodal Model based System to Endow Any Type of Robot with Generative AI](https://mlsanity.com/paper/2401.16205/) (2024)

## Built on by (most cited first, 22 of 22)

- [Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications](https://mlsanity.com/paper/2510.07077/) (2025)
- [Pure Vision Language Action (VLA) Models: A Comprehensive Survey](https://mlsanity.com/paper/2509.19012/) (2025)
- [See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned Aerial Navigation](https://mlsanity.com/paper/2509.22653/) (2025)
- [GRaD-Nav++: Vision-Language Model Enabled Visual Drone Navigation with Gaussian Radiance Fields and Differentiable Dynamics](https://mlsanity.com/paper/2506.14009/) (2025)
- [AerialVLA: A Vision-Language-Action Model for UAV Navigation via Minimalist End-to-End Control](https://mlsanity.com/paper/2603.14363/) (2026)
- [UAV-CodeAgents: Scalable UAV Mission Planning via Multi-Agent ReAct and Vision-Language Reasoning](https://mlsanity.com/paper/2505.07236/) (2025)
- [AIR-VLA: Vision-Language-Action Systems for Aerial Manipulation](https://mlsanity.com/paper/2601.21602/) (2026)
- [APEX: A Decoupled Memory-based Explorer for Asynchronous Aerial Object Goal Navigation](https://mlsanity.com/paper/2602.00551/) (2026)
- [Bring the Apple, Not the Sofa: Impact of Irrelevant Context in Embodied AI Commands on VLA Models](https://mlsanity.com/paper/2510.07067/) (2025)
- [AERMANI-VLM: Structured Prompting and Reasoning for Aerial Manipulation with Vision Language Models](https://mlsanity.com/paper/2511.01472/) (2025)
- [UAV-Track VLA: Embodied Aerial Tracking via Vision-Language-Action Models](https://mlsanity.com/paper/2604.02241/) (2026)
- [ESARBench: A Benchmark for Agentic UAV Embodied Search and Rescue](https://mlsanity.com/paper/2605.01371/) (2026)
- [Embodied4C: Measuring What Matters for Embodied Vision-Language Navigation](https://mlsanity.com/paper/2512.18028/) (2025)
- [CosFly-VLA: A Spatially Aware Vision-Language-Action Model for UAV Tracking](https://mlsanity.com/paper/2607.15004/) (2026)
- [AeroDPO: Unleashing Lightweight UAV Navigation with High-Fidelity Perception and Automated Preference Optimization](https://mlsanity.com/paper/2608.07557/) (2026)
- [HoverAI: An Embodied Aerial Agent for Natural Human-Drone Interaction](https://mlsanity.com/paper/2601.13801/) (2026)
- [DiffusionCinema: Text-to-Aerial Cinematography](https://mlsanity.com/paper/2601.17412/) (2026)
- [PEACE: A Planner-Executor Agent with Constraint Enforcement for UAVs](https://mlsanity.com/paper/2606.00104/) (2026)
- [Vision Language Action (VLA) Models for Unmanned Aerial Robotics and Bimanual Manipulation: A Review](https://mlsanity.com/paper/2607.06706/) (2026)
- [RecoverFly: A Failure-Aware Reinforcement Learning Post-Training Framework for Aerial Vision-Language Navigation](https://mlsanity.com/paper/2608.09467/) (2026)
- [Evaluating Multimodal LLMs as Generalist Vision-Language-Action Agents for Drone Control: Commanding, Approaching, Tracking and Searching](https://mlsanity.com/paper/2609.01404/) (2026)
- [WareFly-VLA: A Vision-Language-Action Framework for UAV Navigation and Human Tracking in Smart Warehouses](https://mlsanity.com/paper/2610.08526/) (2026)

## Abstract (arXiv)

This paper introduces CognitiveDrone, a novel Vision-Language-Action (VLA) model tailored for complex Unmanned Aerial Vehicles (UAVs) tasks that demand advanced cognitive abilities. Trained on a dataset comprising over 8,000 simulated flight trajectories across three key categories-Human Recognition, Symbol Understanding, and Reasoning-the model generates real-time 4D action commands based on first-person visual inputs and textual instructions. To further enhance performance in intricate scenarios, we propose CognitiveDrone-R1, which integrates an additional Vision-Language Model (VLM) reasoning module to simplify task directives prior to high-frequency control. Experimental evaluations using our open-source benchmark, CognitiveDroneBench, reveal that while a racing-oriented model (RaceVLA) achieves an overall success rate of 31.3%, the base CognitiveDrone model reaches 59.6%, and CognitiveDrone-R1 attains a success rate of 77.2%. These results demonstrate improvements of up to 30% in critical cognitive tasks, underscoring the effectiveness of incorporating advanced reasoning capabilities into UAV control systems. Our contributions include the development of a state-of-the-art VLA model for UAV control and the introduction of the first dedicated benchmark for assessing cognitive tasks in drone operations. The complete repository is available at cognitivedrone.github.io