Earth Science Foundation Models: From Perception to Reasoning and Discovery
Not on the map. Why?
Abstract (arXiv)
Large foundation models (FMs) are transforming Earth science by integrating heterogeneous multimodal data, such as multi-platform imagery, gridded reanalysis data, diverse geophysical and geochemical observations, and domain-specific text, to support tasks ranging from basic perception to advanced scientific discovery. This paper provides a unified review of Earth science foundation models (Earth FMs) through two complementary dimensions: depth, which traces the evolution of model capabilities from perception to multimodal reasoning and agentic scientific workflows, and breadth, which summarizes their expanding applications across the atmosphere, hydrosphere, lithosphere, biosphere, anthroposphere, and cryosphere, as well as coupled Earth system processes. Using this framework, we review representative multimodal Earth foundation models and compile more than 200 datasets and benchmarks spanning diverse Earth science tasks and modalities. We further discuss key challenges in multimodal data heterogeneity, scientific reliability and continual updating, scalability and sustainability, and the transition from foundation models to agentic and embodied Earth intelligence, and outline future directions toward more integrated, trustworthy, and actionable AI Earth scientists. Overall, this paper offers a structured roadmap for understanding the development of Earth foundation models from both capability depth and application breadth.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 171 of its 314 references that are in the map. First posted on arXiv 9 May 2026.
Builds on
171 of 314 references.
References · 171
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
- 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 776
- 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensingcited by 363
- 2023ClimaX: A foundation model for weather and climatecited by 337
- 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensingcited by 328
- 2020WeatherBench: A benchmark dataset for data-driven weather forecastingcited by 275
- 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 257
- 2023FuXi: A cascade machine learning forecasting system for 15-day global weather forecastcited by 241
- 2023Neural General Circulation Models for Weather and Climatecited by 239
- 2023WeatherBench 2: A benchmark for the next generation of data-driven global weather modelscited by 236
38 more
- 2023RSGPT: A Remote Sensing Vision Language Model and Benchmarkcited by 207
- 2023SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagerycited by 205
- 2020RSVQA: Visual Question Answering for Remote Sensing Datacited by 203
- 2020CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claimscited by 196
- 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domaincited by 184
- 2020FloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understandingcited by 181
- 2023GenCast: Diffusion-based ensemble forecasting for medium-range weathercited by 177
- 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
- 2022Forecasting Global Weather with Graph Neural Networkscited by 168
- 2022Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecastcited by 166
- 2022Earthformer: Exploring Space-Time Transformers for Earth System Forecastingcited by 152
- 2024SpatialBot: Precise Spatial Understanding with Vision Language Modelscited by 152
- 2023RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensingcited by 151
- 2023SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagerycited by 151
- 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 147
- 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Modelcited by 138
- 2023DiffusionSat: A Generative Foundation Model for Satellite Imagerycited by 131
- 2024ST-LLM: Large Language Models Are Effective Temporal Learnerscited by 126
- 2023SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensingcited by 125
- 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Modelcited by 113
- 2022An Empirical Study of Remote Sensing Pretrainingcited by 105
- 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understandingcited by 104
- 2025TerraMind: Large-Scale Generative Multimodality for Earth Observationcited by 104
- 2024Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generationcited by 100
- 2024From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflectioncited by 98
- 2021ClimateBert: A Pretrained Language Model for Climate-Related Textcited by 90
- 2024EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialoguescited by 85
- 2024UrbanGPT: Spatio-Temporal Large Language Modelscited by 81
- 2024MMEarth: Exploring Multi-Modal Pretext Tasks For Geospatial Representation Learningcited by 77
- 2024AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalitiescited by 74
- 2022AirFormer: Predicting Nationwide Air Quality in China with Transformerscited by 73
- 2023LLMLight: Large Language Models as Traffic Signal Control Agentscited by 73
- 2024TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Datacited by 70
- 2024Foundation Models for Remote Sensing and Earth Observation: A Surveycited by 70
- 2023K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilizationcited by 69
- 2024MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretrainingcited by 66
- 2018InversionNet: A Real-Time and Accurate Full Waveform Inversion with CNNs and continuous CRFscited by 61
- 2024CasCast: Skillful High-resolution Precipitation Nowcasting via Cascaded Modellingcited by 56
50 most cited of 171 All 171 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System
- 2026Strategic Governance of AI Models in Earth Science
- 2025MSEarth: A Multimodal Benchmark for Earth Science Phenomenon Discovery with MLLMs
- 2025OmniEarth-Bench: Towards Holistic Evaluation of Earth's Six Spheres and Cross-Spheres Interactions with Multimodal Observational Earth Data
- 2025Earth AI: Unlocking Geospatial Insights with Foundation Models and Cross-Modal Reasoning
- 2026Emerging Flexible Designs for Geospatial Multimodal Foundation Models
- 2025Towards Scalable and Generalizable Earth Observation Data Mining via Foundation Model Composition
- 2025GeoGrid-Bench: Can Foundation Models Understand Multimodal Gridded Geo-Spatial Data?