# FM-Planner: Foundation Model Guided Path Planning for Autonomous Drone Navigation

arXiv: https://arxiv.org/abs/2505.20783 · HTML: https://mlsanity.com/paper/2505.20783/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 36 references that are in the map.
- First posted on arXiv 27 May 2025.

## Builds on (most cited first, 14 of 14)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Foundation Models in Robotics: Applications, Challenges, and the Future](https://mlsanity.com/paper/2312.07843/) (2023)
- [Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology](https://mlsanity.com/paper/2410.07087/) (2024)
- [Deep Drone Acrobatics](https://mlsanity.com/paper/2006.05768/) (2020)
- [Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation?](https://mlsanity.com/paper/2204.11134/) (2022)
- [EGO-Planner: An ESDF-free Gradient-based Local Planner for Quadrotors](https://mlsanity.com/paper/2008.08835/) (2020)
- [Robust and Efficient Quadrotor Trajectory Generation for Fast Autonomous Flight](https://mlsanity.com/paper/1907.01531/) (2019)
- [Vision-based Learning for Drones: A Survey](https://mlsanity.com/paper/2312.05019/) (2023)
- [Collaborative Target Search with a Visual Drone Swarm: An Adaptive Curriculum Embedded Multistage Reinforcement Learning Approach](https://mlsanity.com/paper/2204.12181/) (2022)
- [Learning Multi-Pursuit Evasion for Safe Targeted Navigation of Drones](https://mlsanity.com/paper/2304.03443/) (2023)
- [Aggressive Quadrotor Flight Using Curiosity-Driven Reinforcement Learning](https://mlsanity.com/paper/2203.14033/) (2022)
- [Learning Resilient Formation Control of Drones with Graph Attention Network](https://mlsanity.com/paper/2409.01953/) (2024)

## Built on by (most cited first, 3 of 3)

- [Agentic Aerial Cinematography: From Dialogue Cues to Cinematic Trajectories](https://mlsanity.com/paper/2509.16176/) (2025)
- [FlightDiffusion: Revolutionising Autonomous Drone Training with Diffusion Models Generating FPV Video](https://mlsanity.com/paper/2509.14082/) (2025)
- [UAVs Meet Embodied Intelligence: Bridging Human Intents and Flying Dynamics Via Harnessing Physical-Digital AI Agents](https://mlsanity.com/paper/2609.18326/) (2026)

## Abstract (arXiv)

Path planning is a critical component in autonomous drone operations, enabling safe and efficient navigation through complex environments. Recent advances in foundation models, particularly large language models (LLMs) and vision-language models (VLMs), have opened new opportunities for enhanced perception and intelligent decision-making in robotics. However, their practical applicability and effectiveness in global path planning remain relatively unexplored. This paper proposes foundation model-guided path planners (FM-Planner) and presents a comprehensive benchmarking study and practical validation for drone path planning. Specifically, we first systematically evaluate eight representative LLM and VLM approaches using standardized simulation scenarios. To enable effective real-time navigation, we then design an integrated LLM-Vision planner that combines semantic reasoning with visual perception. Furthermore, we deploy and validate the proposed path planner through real-world experiments under multiple configurations. Our findings provide valuable insights into the strengths, limitations, and feasibility of deploying foundation models in real-world drone applications and providing practical implementations in autonomous flight. Project site: https://github.com/NTU-ICG/FM-Planner.