# FlexiFly: Interfacing the Physical World with Foundation Models Empowered by Reconfigurable Drone Systems

arXiv: https://arxiv.org/abs/2403.12853 · HTML: https://mlsanity.com/paper/2403.12853/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 57 references that are in the map.
- First posted on arXiv 19 Mar 2024.

## Builds on (most cited first, 8 of 8)

- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [ChatGPT for Robotics: Design Principles and Model Abilities](https://mlsanity.com/paper/2306.17582/) (2023)
- [Text2Motion: From Natural Language Instructions to Feasible Plans](https://mlsanity.com/paper/2303.12153/) (2023)
- [Penetrative AI: Making LLMs Comprehend the Physical World](https://mlsanity.com/paper/2310.09605/) (2023)
- [Transferring Foundation Models for Generalizable Robotic Manipulation](https://mlsanity.com/paper/2306.05716/) (2023)
- [Self-Adapting Large Visual-Language Models to Edge Devices across Visual Modalities](https://mlsanity.com/paper/2403.04908/) (2024)
- [MRS Drone: A Modular Platform for Real-World Deployment of Aerial Multi-Robot Systems](https://mlsanity.com/paper/2306.07229/) (2023)
- [Reconfigurable Drone System for Transportation of Parcels With Variable Mass and Size](https://mlsanity.com/paper/2211.08893/) (2022)

## Built on by (most cited first, 3 of 3)

- [UAVs Meet Agentic AI: A Multidomain Survey of Autonomous Aerial Intelligence and Agentic UAVs](https://mlsanity.com/paper/2506.08045/) (2025)
- [TDBench: A Benchmark for Top-Down Image Understanding with Reliability Analysis of Vision-Language Models](https://mlsanity.com/paper/2504.03748/) (2025)
- [RadioMaster: Multi-Agent System for Autonomous Radio Signal Generation](https://mlsanity.com/paper/2606.01862/) (2026)

## Abstract (arXiv)

Foundation models (FM) have shown immense human-like capabilities for generating digital media. However, foundation models that can freely sense, interact, and actuate the physical domain is far from being realized. This is due to 1) requiring dense deployments of sensors to fully cover and analyze large spaces, while 2) events often being localized to small areas, making it difficult for FMs to pinpoint relevant areas of interest relevant to the current task. We propose FlexiFly, a platform that enables FMs to ``zoom in'' and analyze relevant areas with higher granularity to better understand the physical environment and carry out tasks. FlexiFly accomplishes by introducing 1) a novel image segmentation technique that aids in identifying relevant locations and 2) a modular and reconfigurable sensing and actuation drone platform that FMs can actuate to ``zoom in'' with relevant sensors and actuators. We demonstrate through real smart home deployments that FlexiFly enables FMs and LLMs to complete diverse tasks up to 85% more successfully. FlexiFly is critical step towards FMs and LLMs that can naturally interface with the physical world.