# Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agents

arXiv: https://arxiv.org/abs/2509.23141 · HTML: https://mlsanity.com/paper/2509.23141/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 33 of its 97 references that are in the map.
- First posted on arXiv 27 Sep 2025.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 30 of 33)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning](https://mlsanity.com/paper/2310.09478/) (2023)
- [Kimi K2: Open Agentic Intelligence](https://mlsanity.com/paper/2507.20534/) (2025)
- [Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers](https://mlsanity.com/paper/2409.04109/) (2024)
- [SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models](https://mlsanity.com/paper/2311.07575/) (2023)
- [CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges](https://mlsanity.com/paper/2401.07339/) (2024)
- [Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions](https://mlsanity.com/paper/2503.23278/) (2025)
- [AlphaEarth Foundations: An embedding field model for accurate and efficient global mapping from sparse label data](https://mlsanity.com/paper/2507.22291/) (2025)
- [StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models](https://mlsanity.com/paper/2403.07714/) (2024)
- [AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials](https://mlsanity.com/paper/2412.09605/) (2024)
- [Large Multimodal Agents: A Survey](https://mlsanity.com/paper/2402.15116/) (2024)
- [AI-Researcher: Autonomous Scientific Innovation](https://mlsanity.com/paper/2505.18705/) (2025)
- [VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videos](https://mlsanity.com/paper/2502.01549/) (2025)
- [Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment](https://mlsanity.com/paper/2312.06960/) (2023)
- [ACEBench: Who Wins the Match Point in Tool Usage?](https://mlsanity.com/paper/2501.12851/) (2025)
- [ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflows](https://mlsanity.com/paper/2505.19897/) (2025)
- [RS-Agent: Automating Remote Sensing Tasks through Intelligent Agent](https://mlsanity.com/paper/2406.07089/) (2024)
- [ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks](https://mlsanity.com/paper/2505.23752/) (2025)
- [EarthNets: Empowering AI in Earth Observation](https://mlsanity.com/paper/2210.04936/) (2022)
- [From Mind to Machine: The Rise of Manus AI as a Fully Autonomous Digital Agent](https://mlsanity.com/paper/2505.02024/) (2025)
- [SM3Det: A Unified Model for Multi-Modal Remote Sensing Object Detection](https://mlsanity.com/paper/2412.20665/) (2024)
- [Surveying the MLLM Landscape: A Meta-Review of Current Surveys](https://mlsanity.com/paper/2409.18991/) (2024)
- [RingMo-Agent: A Unified Remote Sensing Foundation Model for Multi-Platform and Multi-Modal Reasoning](https://mlsanity.com/paper/2507.20776/) (2025)
- [Towards LLM Agents for Earth Observation](https://mlsanity.com/paper/2504.12110/) (2025)

## Built on by (most cited first, 30 of 46)

- [Designing Domain-Specific Agents via Hierarchical Task Abstraction Mechanism](https://mlsanity.com/paper/2511.17198/) (2025)
- [OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agents](https://mlsanity.com/paper/2602.17665/) (2026)
- [GeoZero: Incentivizing Reasoning from Scratch on Geospatial Scenes](https://mlsanity.com/paper/2511.22645/) (2025)
- [BLINK-Twice: You see, but do you observe? A Reasoning Benchmark on Visual Perception](https://mlsanity.com/paper/2510.09361/) (2025)
- [OpenEarth-Agent: From Tool Calling to Tool Creation for Open-Environment Earth Observation](https://mlsanity.com/paper/2603.22148/) (2026)
- [CangLing-KnowFlow: A Unified Knowledge-and-Flow-fused Agent for Comprehensive Remote Sensing Applications](https://mlsanity.com/paper/2512.15231/) (2025)
- [RemoteAgent: Bridging Vague Human Intents and Earth Observation with RL-based Agentic MLLMs](https://mlsanity.com/paper/2604.07765/) (2026)
- [TerraScope: Pixel-Grounded Visual Reasoning for Earth Observation](https://mlsanity.com/paper/2603.19039/) (2026)
- [Agentic AI in Remote Sensing: Foundations, Taxonomy, and Emerging Systems](https://mlsanity.com/paper/2601.01891/) (2026)
- [Physically Interpretable AlphaEarth Foundation Model Embeddings Enable LLM-Based Land Surface Intelligence](https://mlsanity.com/paper/2602.10354/) (2026)
- [Can LLM Agents Respond to Disasters? Benchmarking Heterogeneous Geospatial Reasoning in Emergency Operations](https://mlsanity.com/paper/2605.11633/) (2026)
- [GeoAgentBench: A Dynamic Execution Benchmark for Tool-Augmented Agents in Spatial Analysis](https://mlsanity.com/paper/2604.13888/) (2026)
- [TerraBench: Can Agents Reason Over Heterogeneous Earth-System Data?](https://mlsanity.com/paper/2606.13148/) (2026)
- [CausalGame: Benchmarking Causal Thinking of LLM Agents in Games](https://mlsanity.com/paper/2607.04293/) (2026)
- [Characterizing AlphaEarth Embedding Geometry for Agentic Environmental Reasoning](https://mlsanity.com/paper/2604.18715/) (2026)
- [CausalEvolve: Towards Open-Ended Discovery with Causal Scratchpad](https://mlsanity.com/paper/2603.14575/) (2026)
- [RS-Claw: Progressive Active Tool Exploration via Hierarchical Skill Trees for Remote Sensing Agents](https://mlsanity.com/paper/2605.13391/) (2026)
- [TerraLogic: A Benchmark for Hierarchical Geospatial Reasoning in Earth Observation](https://mlsanity.com/paper/2607.12497/) (2026)
- [IC-EO: Interpretable Code-based assistant for Earth Observation](https://mlsanity.com/paper/2602.00117/) (2026)
- [Agentic AI for Remote Sensing: Technical Challenges and Research Directions](https://mlsanity.com/paper/2604.24919/) (2026)
- [SCI-PRM: A Tool Aware Process Reward Model for Scientific Reasoning Verification](https://mlsanity.com/paper/2606.04579/) (2026)
- [RSMeM: Knowledge-Enhanced Memory Evolution for Remote Sensing Agents with Systematic Evaluation](https://mlsanity.com/paper/2607.24772/) (2026)
- [RingMoClaw: An Experience-Inspired Multi-Agent Framework for Self-Evolving Research in Remote Sensing](https://mlsanity.com/paper/2609.00814/) (2026)
- [EO-Gym: A Multimodal, Interactive Environment for Earth Observation Agents](https://mlsanity.com/paper/2605.01250/) (2026)
- [Bidirectional Semantic Complementary Tool Retrieval for Remote Sensing Agents](https://mlsanity.com/paper/2606.07538/) (2026)
- [TianJi-Environ: An Autonomous AI Scientist for Atmospheric Environmental Research](https://mlsanity.com/paper/2606.07697/) (2026)
- [GeoForge: Non-Parametric Self-Evolving Agents for Earth-Observation Reasoning](https://mlsanity.com/paper/2608.10494/) (2026)
- [S2SServiceBench: A Multimodal Benchmark for Last-Mile S2S Climate Services](https://mlsanity.com/paper/2602.14017/) (2026)
- [Earth Science Foundation Models: From Perception to Reasoning and Discovery](https://mlsanity.com/paper/2605.12542/) (2026)
- [AgroTools: A Benchmark for Tool-Augmented Multimodal Agents in Agriculture](https://mlsanity.com/paper/2605.22366/) (2026)

## Abstract (arXiv)

Earth observation (EO) is essential for understanding the evolving states of the Earth system. Although recent MLLMs have advanced EO research, they still lack the capability to tackle complex tasks that require multi-step reasoning and the use of domain-specific tools. Agent-based methods offer a promising direction, but current attempts remain in their infancy, confined to RGB perception, shallow reasoning, and lacking systematic evaluation protocols. To overcome these limitations, we introduce Earth-Agent, the first agentic framework that unifies RGB and spectral EO data within an MCP-based tool ecosystem, enabling cross-modal, multi-step, and quantitative spatiotemporal reasoning beyond pretrained MLLMs. Earth-Agent supports complex scientific tasks such as geophysical parameter retrieval and quantitative spatiotemporal analysis by dynamically invoking expert tools and models across modalities. To support comprehensive evaluation, we further propose Earth-Bench, a benchmark of 248 expert-curated tasks with 13,729 images, spanning spectrum, products and RGB modalities, and equipped with a dual-level evaluation protocol that assesses both reasoning trajectories and final outcomes. We conduct comprehensive experiments varying different LLM backbones, comparisons with general agent frameworks, and comparisons with MLLMs on remote sensing benchmarks, demonstrating both the effectiveness and potential of Earth-Agent. Earth-Agent establishes a new paradigm for EO analysis, moving the field toward scientifically grounded, next-generation applications of LLMs in Earth observation. More information about Earth-Agent can be found at https://github.com/opendatalab/Earth-Agent