Risk-Aware LLM Agents for Geospatial Data Retrieval: Design and Preliminary Adversarial Evaluation
On the map: LLM agents and tool use → Multimodal and embodied agents
also in 1 more
Abstract (arXiv)
We present an LLM-driven framework for retrieving remote sensing data from cloud-based geospatial catalogues using natural language queries. The system converts user intent into structured API calls, enabling efficient access to satellite imagery and environmental datasets. The architecture integrates three agents: Guardrail for safety and policy enforcement, General-QA for intent interpretation, and Recommender-Analyst for schema-aware API call generation. This coordinated design ensures reliable, semantically aligned interaction with external data services. The modular framework is portable across platforms through API schema substitution and supports applications in environmental monitoring, disaster response, and climate analysis. It establishes a scalable interface between user intent and geospatial infrastructure, enabling streamlined and automated Earth observation workflows. Preliminary experiments under adversarial multi-turn settings show that prompt-level safety instructions improve robustness, although rare high-impact failures persist in API manipulation scenarios and highlight the need for adaptive, system-level defenses that balance safety, usability, and cost efficiency, which motivates the use of our intercept-level Guardrail agent.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 9 of its 14 references that are in the map. First posted on arXiv 13 Jun 2026.
Builds on
9 of 14 references.
References · 9
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
- 2025ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrailscited by 29
- 2025MrGuard: A Multilingual Reasoning Guardrail for Universal LLM Safetycited by 9
- 2025Instructor-Worker Large Language Model System for Policy Recommendation: a Case Study on Air Quality Analysis of the January 2025 Los Angeles Wildfirescited by 6
- 2025Digital Twin Buildings: 3D Modeling, GIS Integration, and Visual Descriptions Using Gaussian Splatting, ChatGPT/Deepseek, and Google Maps Platformcited by 4
- 2024GEE-OPs: An Operator Knowledge Base for Geospatial Code Generation on the Google Earth Engine Platform Powered by Large Language Modelscited by 3
- 2025SGuard-v1: Safety Guardrail for Large Language Modelscited by 3
- 2025Evaluating the Robustness of Large Language Model Safety Guardrails Against Adversarial Attackscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Empowering LLM Agents with Geospatial Awareness: Toward Grounded Reasoning for Wildfire Response
- 2026GeoDisaster: Benchmarking Orchestrated Agents for Operational Disaster Geo-Intelligence
- 2025ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks
- 2025GeoJSON Agents:A Multi-Agent LLM Architecture for Geospatial Analysis-Function Calling vs Code Generation
- 2026OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agents
- 2025Geo-OLM: Enabling Sustainable Earth Observation Studies with Cost-Efficient Open Language Models & State-Driven Workflows
- 2024Evaluating Tool-Augmented Agents in Remote Sensing Platforms
- 2026RSMeM: Knowledge-Enhanced Memory Evolution for Remote Sensing Agents with Systematic Evaluation