mlsanityData to 7 Oct 2026

Deploying Foundation Model-Enabled Air and Ground Robots in the Field: Challenges and Opportunities

arXiv 2505.09477 ↗First posted 14 May 2025cs.RO cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

The integration of foundation models (FMs) into robotics has enabled robots to understand natural language and reason about the semantics in their environments. However, existing FM-enabled robots primary operate in closed-world settings, where the robot is given a full prior map or has a full view of its workspace. This paper addresses the deployment of FM-enabled robots in the field, where missions often require a robot to operate in large-scale and unstructured environments. To effectively accomplish these missions, robots must actively explore their environments, navigate obstacle-cluttered terrain, handle unexpected sensor inputs, and operate with compute constraints. We discuss recent deployments of SPINE, our LLM-enabled autonomy framework, in field robotic settings. To the best of our knowledge, we present the first demonstration of large-scale LLM-enabled robot planning in unstructured environments with several kilometers of missions. SPINE is agnostic to a particular LLM, which allows us to distill small language models capable of running onboard size, weight and power (SWaP) limited platforms. Via preliminary model distillation work, we then present the first language-driven UAV planner using on-device language models. We conclude our paper by proposing several promising directions for future research.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 33 of its 39 references that are in the map. First posted on arXiv 14 May 2025.

Builds on

33 of 39 references.

References · 33

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  4. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  5. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  6. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  7. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  8. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  9. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  10. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  11. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  12. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
21 more
  1. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
  2. 2020RoboTHOR: An Open Simulation-to-Real Embodied AI Platformcited by 190
  3. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 117
  4. 2023Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planningcited by 88
  5. 2024HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigationcited by 71
  6. 2024Language-Grounded Dynamic Scene Graphs for Interactive Object Search with Mobile Manipulationcited by 69
  7. 2022ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hintscited by 55
  8. 2024Driving Everywhere with Large Language Model Policy Adaptationcited by 53
  9. 2023Deploying and Evaluating LLMs to Program Service Mobile Robotscited by 36
  10. 2021NeBula: Quest for Robotic Autonomy in Challenging Environments; TEAM CoSTAR at the DARPA Subterranean Challengecited by 32
  11. 2024RoboHop: Segment-based Topological Map Representation for Open-World Visual Navigationcited by 25
  12. 2024Exploring Spatial Representation to Enhance LLM Reasoning in Aerial Vision-Language Navigationcited by 18
  13. 2024SPINE: Online Semantic Planning for Missions with Incomplete Natural Language Specifications in Unstructured Environmentscited by 16
  14. 2024GroundGrid:LiDAR Point Cloud Ground Segmentation and Terrain Estimationcited by 14
  15. 2023Reasoning about the Unseen for Efficient Outdoor Object Navigationcited by 13
  16. 2021Large-scale Autonomous Flight with Real-time Semantic SLAM under Dense Forest Canopycited by 12
  17. 2022Stronger Together: Air-Ground Robotic Collaboration Using Semanticscited by 8
  18. 2024NEUSIS: A Compositional Neuro-Symbolic Framework for Autonomous Perception, Reasoning, and Planning in Complex UAV Search Missionscited by 8
  19. 2023Enabling Large-scale Heterogeneous Collaboration with Opportunistic Communicationscited by 4
  20. 2024EvMAPPER: High Altitude Orthomapping with Event Camerascited by 4
  21. 2024Proprioception Is All You Need: Terrain Classification for Boreal Forestscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Heterogeneous Robot Collaboration in Unstructured Environments with Grounded Generative Intelligencecited by 3
  2. 2026CoFL: Continuous Flow Fields for Language-Conditioned Navigationcited by 2
  3. 2025CoFineLLM: Conformal Finetuning of LLMs for Language-Instructed Robot Planningcited by 0