mlsanityData to 14 Oct 2026

Earth Science Foundation Models: From Perception to Reasoning and Discovery

arXiv 2605.12542 ↗First posted 9 May 2026astro-ph.IM astro-ph.EP cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Large foundation models (FMs) are transforming Earth science by integrating heterogeneous multimodal data, such as multi-platform imagery, gridded reanalysis data, diverse geophysical and geochemical observations, and domain-specific text, to support tasks ranging from basic perception to advanced scientific discovery. This paper provides a unified review of Earth science foundation models (Earth FMs) through two complementary dimensions: depth, which traces the evolution of model capabilities from perception to multimodal reasoning and agentic scientific workflows, and breadth, which summarizes their expanding applications across the atmosphere, hydrosphere, lithosphere, biosphere, anthroposphere, and cryosphere, as well as coupled Earth system processes. Using this framework, we review representative multimodal Earth foundation models and compile more than 200 datasets and benchmarks spanning diverse Earth science tasks and modalities. We further discuss key challenges in multimodal data heterogeneity, scientific reliability and continual updating, scalability and sustainability, and the transition from foundation models to agentic and embodied Earth intelligence, and outline future directions toward more integrated, trustworthy, and actionable AI Earth scientists. Overall, this paper offers a structured roadmap for understanding the development of Earth foundation models from both capability depth and application breadth.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 171 of its 314 references that are in the map. First posted on arXiv 9 May 2026.

Builds on

171 of 314 references.

References · 171

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  4. 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 776
  5. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensingcited by 363
  6. 2023ClimaX: A foundation model for weather and climatecited by 337
  7. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensingcited by 328
  8. 2020WeatherBench: A benchmark dataset for data-driven weather forecastingcited by 275
  9. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 257
  10. 2023FuXi: A cascade machine learning forecasting system for 15-day global weather forecastcited by 241
  11. 2023Neural General Circulation Models for Weather and Climatecited by 239
  12. 2023WeatherBench 2: A benchmark for the next generation of data-driven global weather modelscited by 236
38 more
  1. 2023RSGPT: A Remote Sensing Vision Language Model and Benchmarkcited by 207
  2. 2023SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagerycited by 205
  3. 2020RSVQA: Visual Question Answering for Remote Sensing Datacited by 203
  4. 2020CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claimscited by 196
  5. 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domaincited by 184
  6. 2020FloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understandingcited by 181
  7. 2023GenCast: Diffusion-based ensemble forecasting for medium-range weathercited by 177
  8. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
  9. 2022Forecasting Global Weather with Graph Neural Networkscited by 168
  10. 2022Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecastcited by 166
  11. 2022Earthformer: Exploring Space-Time Transformers for Earth System Forecastingcited by 152
  12. 2024SpatialBot: Precise Spatial Understanding with Vision Language Modelscited by 152
  13. 2023RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensingcited by 151
  14. 2023SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagerycited by 151
  15. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 147
  16. 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Modelcited by 138
  17. 2023DiffusionSat: A Generative Foundation Model for Satellite Imagerycited by 131
  18. 2024ST-LLM: Large Language Models Are Effective Temporal Learnerscited by 126
  19. 2023SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensingcited by 125
  20. 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Modelcited by 113
  21. 2022An Empirical Study of Remote Sensing Pretrainingcited by 105
  22. 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understandingcited by 104
  23. 2025TerraMind: Large-Scale Generative Multimodality for Earth Observationcited by 104
  24. 2024Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generationcited by 100
  25. 2024From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflectioncited by 98
  26. 2021ClimateBert: A Pretrained Language Model for Climate-Related Textcited by 90
  27. 2024EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialoguescited by 85
  28. 2024UrbanGPT: Spatio-Temporal Large Language Modelscited by 81
  29. 2024MMEarth: Exploring Multi-Modal Pretext Tasks For Geospatial Representation Learningcited by 77
  30. 2024AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalitiescited by 74
  31. 2022AirFormer: Predicting Nationwide Air Quality in China with Transformerscited by 73
  32. 2023LLMLight: Large Language Models as Traffic Signal Control Agentscited by 73
  33. 2024TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Datacited by 70
  34. 2024Foundation Models for Remote Sensing and Earth Observation: A Surveycited by 70
  35. 2023K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilizationcited by 69
  36. 2024MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretrainingcited by 66
  37. 2018InversionNet: A Real-Time and Accurate Full Waveform Inversion with CNNs and continuous CRFscited by 61
  38. 2024CasCast: Skillful High-resolution Precipitation Nowcasting via Cascaded Modellingcited by 56

50 most cited of 171 All 171 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System
  2. 2026Strategic Governance of AI Models in Earth Science
  3. 2025MSEarth: A Multimodal Benchmark for Earth Science Phenomenon Discovery with MLLMs
  4. 2025OmniEarth-Bench: Towards Holistic Evaluation of Earth's Six Spheres and Cross-Spheres Interactions with Multimodal Observational Earth Data
  5. 2025Earth AI: Unlocking Geospatial Insights with Foundation Models and Cross-Modal Reasoning
  6. 2026Emerging Flexible Designs for Geospatial Multimodal Foundation Models
  7. 2025Towards Scalable and Generalizable Earth Observation Data Mining via Foundation Model Composition
  8. 2025GeoGrid-Bench: Can Foundation Models Understand Multimodal Gridded Geo-Spatial Data?