# AeroDPO: Unleashing Lightweight UAV Navigation with High-Fidelity Perception and Automated Preference Optimization

arXiv: https://arxiv.org/abs/2608.07557 · HTML: https://mlsanity.com/paper/2608.07557/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 28 of its 37 references that are in the map.
- First posted on arXiv 2 Aug 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Reward modeling and RLHF](https://mlsanity.com/alignment/reward-modeling-and-rlhf/)

## Builds on (most cited first, 28 of 28)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Learning by Cheating](https://mlsanity.com/paper/1912.12294/) (2019)
- [Safe Exploration in Continuous Action Spaces](https://mlsanity.com/paper/1801.08757/) (2018)
- [Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones](https://mlsanity.com/paper/2010.15920/) (2020)
- [RUDDER: Return Decomposition for Delayed Rewards](https://mlsanity.com/paper/1806.07857/) (2018)
- [AerialVLN: Vision-and-Language Navigation for UAVs](https://mlsanity.com/paper/2308.06735/) (2023)
- [Defensive Quantization: When Efficiency Meets Robustness](https://mlsanity.com/paper/1904.08444/) (2019)
- [Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology](https://mlsanity.com/paper/2410.07087/) (2024)
- [UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobility](https://mlsanity.com/paper/2501.02341/) (2025)
- [CityNavAgent: Aerial Vision-and-Language Navigation with Hierarchical Semantic Planning and Global Memory](https://mlsanity.com/paper/2505.05622/) (2025)
- [UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learning](https://mlsanity.com/paper/2505.15725/) (2025)
- [RaceVLA: VLA-based Racing Drone Navigation with Human-like Behaviour](https://mlsanity.com/paper/2503.02572/) (2025)
- [VLA-AN: An Efficient and Onboard Vision-Language-Action Framework for Aerial Navigation in Complex Environments](https://mlsanity.com/paper/2512.15258/) (2025)
- [CognitiveDrone: A VLA Model and Evaluation Benchmark for Real-Time Cognitive Task Solving and Reasoning in UAVs](https://mlsanity.com/paper/2503.01378/) (2025)
- [See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned Aerial Navigation](https://mlsanity.com/paper/2509.22653/) (2025)
- [LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integration](https://mlsanity.com/paper/2512.22010/) (2025)
- [UAVs Meet Agentic AI: A Multidomain Survey of Autonomous Aerial Intelligence and Agentic UAVs](https://mlsanity.com/paper/2506.08045/) (2025)
- [IndoorUAV: Benchmarking Vision-Language UAV Navigation in Continuous Indoor Environments](https://mlsanity.com/paper/2512.19024/) (2025)
- [Vision-and-Language Navigation for UAVs: Progress, Challenges, and a Research Roadmap](https://mlsanity.com/paper/2604.13654/) (2026)
- [Neural Potential Field for Obstacle-Aware Local Motion Planning](https://mlsanity.com/paper/2310.16362/) (2023)
- [See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of View](https://mlsanity.com/paper/2606.20045/) (2026)
- [Prompt-responsive Object Retrieval with Memory-augmented Student-Teacher Learning](https://mlsanity.com/paper/2505.02232/) (2025)
- [Deployable Vision-driven UAV River Navigation via Human-in-the-loop Preference Alignment](https://mlsanity.com/paper/2511.01083/) (2025)
- [Mitigating Error Accumulation in Continuous Navigation via Memory-Augmented Kalman Filtering](https://mlsanity.com/paper/2602.11183/) (2026)

## Built on by (most cited first, 1 of 1)

- [AVERT-VLN: Abstention-aware Visual Error Recovery and Training for Vision-and-Language Navigation](https://mlsanity.com/paper/2609.39579/) (2026)

## Abstract (arXiv)

Vision-Language Navigation for Unmanned Aerial Vehicles (UAV-VLN) requires rapid and reactive control in complex 3D environments. Recent minimalist end-to-end paradigms show great promise but typically rely on massive language models containing billions of parameters, incurring prohibitive latency for real-world edge deployment. In this paper, we challenge this parameter-heavy reliance. Comprehensive cross-scale evaluations reveal the critical insight that perception quality fundamentally outweighs language reasoning capacity. We demonstrate that a lightweight 2B model equipped with high-fidelity visual inputs completely matches the overall success rates of massive 7B baselines. However, this minimalist policy exposes a fundamental robustness flaw inherent to pure Behavior Cloning (BC). Lacking explicit negative feedback, the agent fails to internalize robust spatial constraints and exhibits alarming collision rates in out-of-distribution (OOD) scenarios. To overcome this vulnerability without relying on unscalable human annotations, we propose AeroDPO, a zero-cost automated Direct Preference Optimization pipeline driven by deterministic physical simulation state rollback. Upon detecting collisions, the system autonomously rewinds the environment to extract causal reasoning errors as rejected actions, applies decoupled privileged interventions to synthesize collision-avoidance preferred maneuvers, and leverages an offline vision language inspector to filter visual ambiguities. By equipping our 2B model with this automated data flywheel, AeroDPO boosts success rates to 49.16% on unmapped scenarios while drastically suppressing collision rates, establishing a new SOTA for autonomous aerial agents.