SwarmGPT: Combining Large Language Models with Safe Motion Planning for Drone Swarm Choreography
Not on the map. Why?
Abstract (arXiv)
Drone swarm performances -- synchronized, expressive aerial displays set to music -- have emerged as a captivating application of modern robotics. Yet designing smooth, safe choreographies remains a complex task requiring expert knowledge. We present SwarmGPT, a language-based choreographer that leverages the reasoning power of large language models (LLMs) to streamline drone performance design. The LLM is augmented by a safety filter that ensures deployability by making minimal corrections when safety or feasibility constraints are violated. By decoupling high-level choreographic design from low-level motion planning, our system enables non-experts to iteratively refine choreographies using natural language without worrying about collisions or actuator limits. We validate our approach through simulations with swarms up to 200 drones and real-world experiments with up to 20 drones performing choreographies to diverse types of songs, demonstrating scalable, synchronized, and safe performances. Beyond entertainment, this work offers a blueprint for integrating foundation models into safety-critical swarm robotics applications.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 23 of its 48 references that are in the map. First posted on arXiv 11 Dec 2024.
Builds on
23 of 48 references.
References · 23
- 2023GPT-4 Technical Reportcited by 18,132
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
- 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
- 2022Visual Language Maps for Robot Navigationcited by 362
- 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
- 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
- 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 296
- 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 212
11 more
- 2023Physically Grounded Vision-Language Models for Robotic Manipulationcited by 163
- 2023Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisitioncited by 156
- 2023Language Embedded Radiance Fields for Zero-Shot Task-Oriented Graspingcited by 91
- 2021Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Controlcited by 68
- 2022LATTE: LAnguage Trajectory TransformErcited by 40
- 2023SayTap: Language to Quadrupedal Locomotioncited by 40
- 2019Online Trajectory Generation with Distributed Model Predictive Control for Multi-Robot Motion Planningcited by 13
- 2023AMSwarm: An Alternating Minimization Approach for Safe Motion Planning of Quadrotor Swarms in Cluttered Environmentscited by 5
- 2018Trajectory Generation for Multiagent Point-To-Point Transitions via Distributed Model Predictive Controlcited by 4
- 2017Downwash-Aware Trajectory Planning for Large Quadrotor Teamscited by 2
- 2018Fast and In Sync: Periodic Swarm Patterns for Quadrotorscited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 4.
5 papers.
- 2025TACOS: Task Agnostic COordinator of a multi-drone Systemcited by 2
- 2026Crazyflow: An Accurate, GPU-Accelerated, Differentiable Drone Simulator in JAXcited by 1
- 2026RT-SHCUA: Real-Time Self-Hosted Computer-Use Agent for UAV Controlcited by 0
- 2026Large Language Models in the Loop: A Stability- and Network-Aware Survey in Networked Control, Cyber-Physical, and Multi-Agent Systemscited by 0
- 2026Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directionscited by 0
- 2026Large Language Models in the Loop: A Stability- and Network-Aware Survey in Networked Control, Cyber-Physical, and Multi-Agent Systems15 Sep 2026
- 2026Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directions19 Aug 2026
- 2026RT-SHCUA: Real-Time Self-Hosted Computer-Use Agent for UAV Control20 Jul 2026
- 2026Crazyflow: An Accurate, GPU-Accelerated, Differentiable Drone Simulator in JAX31 May 2026
- 2025TACOS: Task Agnostic COordinator of a multi-drone System2 Oct 2025
Similar, not cited
Close in wording, no citation either way
- 2026SkySim: A ROS2-based Simulation Environment for Natural Language Control of Drone Swarms using Large Language Models
- 2026AeroWeaver: An Embodied-Agent Harness for Weaving Aerial Skills into Distributed, Adaptive Swarm Execution
- 2025SINGER: An Onboard Generalist Vision-Language Navigation Policy for Drones
- 2026Say the Mission, Execute the Swarm: Agent-Enhanced LLM Reasoning in the Web-of-Drones
- 2025Using Language Models as Closed-Loop High-Level Planners for Robotics Applications: A Brief Overview and Benchmarks
- 2025Multimodal Large Language Models-Enabled UAV Swarm: Towards Efficient and Intelligent Autonomous Aerial Systems
- 2026Evaluating Multimodal LLMs as Generalist Vision-Language-Action Agents for Drone Control: Commanding, Approaching, Tracking and Searching
- 2026Melding LLM and temporal logic for reliable human-swarm collaboration in complex scenarios ↗