# On the Foundations of Earth and Climate Foundation Models

arXiv: https://arxiv.org/abs/2405.04285 · HTML: https://mlsanity.com/paper/2405.04285/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 88 of its 134 references that are in the map.
- First posted on arXiv 7 May 2024.

## Builds on (most cited first, 30 of 88)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model](https://mlsanity.com/paper/2401.09417/) (2024)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [CogVLM: Visual Expert for Pretrained Language Models](https://mlsanity.com/paper/2311.03079/) (2023)
- [Scaling Vision Transformers to 22 Billion Parameters](https://mlsanity.com/paper/2302.05442/) (2023)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [Does Learning Require Memorization? A Short Tale about a Long Tail](https://mlsanity.com/paper/1906.05271/) (2019)
- [SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery](https://mlsanity.com/paper/2207.08051/) (2022)
- [RemoteCLIP: A Vision Language Foundation Model for Remote Sensing](https://mlsanity.com/paper/2306.11029/) (2023)
- [ClimaX: A foundation model for weather and climate](https://mlsanity.com/paper/2301.10343/) (2023)
- [Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning](https://mlsanity.com/paper/2212.14532/) (2022)
- [WeatherBench: A benchmark dataset for data-driven weather forecasting](https://mlsanity.com/paper/2002.00469/) (2020)
- [FuXi: A cascade machine learning forecasting system for 15-day global weather forecast](https://mlsanity.com/paper/2306.12873/) (2023)
- [WeatherBench 2: A benchmark for the next generation of data-driven global weather models](https://mlsanity.com/paper/2308.15560/) (2023)
- [BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understanding](https://mlsanity.com/paper/1902.06148/) (2019)
- [Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere](https://mlsanity.com/paper/2306.03838/) (2023)
- [Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data](https://mlsanity.com/paper/2103.16607/) (2021)
- [RSGPT: A Remote Sensing Vision Language Model and Benchmark](https://mlsanity.com/paper/2307.15266/) (2023)
- [SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery](https://mlsanity.com/paper/2312.10115/) (2023)

## Built on by (most cited first, 30 of 33)

- [Towards a Unified Copernicus Foundation Model for Earth Vision](https://mlsanity.com/paper/2503.11849/) (2025)
- [Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretraining](https://mlsanity.com/paper/2405.20462/) (2024)
- [WeatherQA: Can Multimodal Language Models Reason about Severe Weather?](https://mlsanity.com/paper/2406.11217/) (2024)
- [Multi-Agent Geospatial Copilots for Remote Sensing Workflows](https://mlsanity.com/paper/2501.16254/) (2025)
- [GEO-Bench-2: From Performance to Capability, Rethinking Evaluation in Geospatial AI](https://mlsanity.com/paper/2511.15658/) (2025)
- [Deep Learning and Foundation Models for Weather Prediction: A Survey](https://mlsanity.com/paper/2501.06907/) (2025)
- [REOBench: Benchmarking Robustness of Earth Observation Foundation Models](https://mlsanity.com/paper/2505.16793/) (2025)
- [Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodality](https://mlsanity.com/paper/2603.00988/) (2026)
- [Geo-OLM: Enabling Sustainable Earth Observation Studies with Cost-Efficient Open Language Models & State-Driven Workflows](https://mlsanity.com/paper/2504.04319/) (2025)
- [AgriFM: A Multi-source Temporal Remote Sensing Foundation Model for Agriculture Mapping](https://mlsanity.com/paper/2505.21357/) (2025)
- [Evaluating and Benchmarking Foundation Models for Earth Observation and Geospatial AI](https://mlsanity.com/paper/2406.18295/) (2024)
- [Paving the way toward foundation models for irregular and unaligned Satellite Image Time Series](https://mlsanity.com/paper/2407.08448/) (2024)
- [Physically Interpretable AlphaEarth Foundation Model Embeddings Enable LLM-Based Land Surface Intelligence](https://mlsanity.com/paper/2602.10354/) (2026)
- [Machine Learning for Methane Detection and Quantification from Space - A survey](https://mlsanity.com/paper/2408.15122/) (2024)
- [ExEBench: Benchmarking Foundation Models on Extreme Earth Events](https://mlsanity.com/paper/2505.08529/) (2025)
- [Foundation Models for Environmental Science: A Survey of Emerging Frontiers](https://mlsanity.com/paper/2504.04280/) (2025)
- [A Genealogy of Foundation Models in Remote Sensing](https://mlsanity.com/paper/2504.17177/) (2025)
- [Characterizing AlphaEarth Embedding Geometry for Agentic Environmental Reasoning](https://mlsanity.com/paper/2604.18715/) (2026)
- [Beyond Grid Data: Exploring Graph Neural Networks for Earth Observation](https://mlsanity.com/paper/2411.03223/) (2024)
- [Daily Land Surface Temperature Reconstruction in Landsat Cross-Track Areas Using Deep Ensemble Learning With Uncertainty Quantification](https://mlsanity.com/paper/2502.14433/) (2025)
- [A Survey of Foundation Models for Environmental Science](https://mlsanity.com/paper/2503.03142/) (2025)
- [Resolution Revolution: A Physics-Guided Deep Learning Framework for Spatiotemporal Temperature Reconstruction](https://mlsanity.com/paper/2507.09872/) (2025)
- [Field-Space Attention for Structure-Preserving Earth System Transformers](https://mlsanity.com/paper/2512.20350/) (2025)
- [YieldSAT: A Multimodal Benchmark Dataset for High-Resolution Crop Yield Prediction](https://mlsanity.com/paper/2604.00940/) (2026)
- [Heterogeneous Scientific Foundation Model Collaboration](https://mlsanity.com/paper/2604.27351/) (2026)
- [SHAZAM: Self-Supervised Change Monitoring for Hazard Detection and Mapping](https://mlsanity.com/paper/2503.00348/) (2025)
- [On the use of Graphs for Satellite Image Time Series](https://mlsanity.com/paper/2505.16685/) (2025)
- [Geospatial Machine Learning Libraries](https://mlsanity.com/paper/2510.02572/) (2025)
- [Adaptive Gradient Calibration for Single-Positive Multi-Label Learning in Remote Sensing Image Scene Classification](https://mlsanity.com/paper/2510.08269/) (2025)
- [LandSegmenter: Towards a Flexible Foundation Model for Land Use and Land Cover Mapping](https://mlsanity.com/paper/2511.08156/) (2025)

## Abstract (arXiv)

Foundation models have enormous potential in advancing Earth and climate sciences, however, current approaches may not be optimal as they focus on a few basic features of a desirable Earth and climate foundation model. Crafting the ideal Earth foundation model, we define eleven features which would allow such a foundation model to be beneficial for any geoscientific downstream application in an environmental- and human-centric manner.We further shed light on the way forward to achieve the ideal model and to evaluate Earth foundation models. What comes after foundation models? Energy efficient adaptation, adversarial defenses, and interpretability are among the emerging directions.