# WeatherGFM: Learning A Weather Generalist Foundation Model via In-context Learning

arXiv: https://arxiv.org/abs/2411.05420 · HTML: https://mlsanity.com/paper/2411.05420/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 59 references that are in the map.
- First posted on arXiv 8 Nov 2024.

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [ClimaX: A foundation model for weather and climate](https://mlsanity.com/paper/2301.10343/) (2023)
- [FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead](https://mlsanity.com/paper/2304.02948/) (2023)
- [ClimateBert: A Pretrained Language Model for Climate-Related Text](https://mlsanity.com/paper/2110.12010/) (2021)
- [HAT: Hybrid Attention Transformer for Image Restoration](https://mlsanity.com/paper/2309.05239/) (2023)
- [DiffDA: a Diffusion Model for Weather-scale Data Assimilation](https://mlsanity.com/paper/2401.05932/) (2024)
- [CasCast: Skillful High-resolution Precipitation Nowcasting via Cascaded Modelling](https://mlsanity.com/paper/2402.04290/) (2024)
- [OceanGPT: A Large Language Model for Ocean Science Tasks](https://mlsanity.com/paper/2310.02031/) (2023)
- [Prompt Federated Learning for Weather Forecasting: Toward Foundation Models on Meteorological Data](https://mlsanity.com/paper/2301.09152/) (2023)
- [FengWu-GHR: Learning the Kilometer-scale Medium-range Global Weather Forecasting](https://mlsanity.com/paper/2402.00059/) (2024)
- [W-MAE: Pre-trained weather model with masked autoencoder for multi-variable weather forecasting](https://mlsanity.com/paper/2304.08754/) (2023)
- [Unifying Image Processing as Visual Prompting Question Answering](https://mlsanity.com/paper/2310.10513/) (2023)
- [Towards LLM-driven Dialogue State Tracking](https://mlsanity.com/paper/2310.14970/) (2023)
- [A Foundation Model for the Earth System](https://mlsanity.com/paper/2405.13063/) (2024)
- [Finding Visual Task Vectors](https://mlsanity.com/paper/2404.05729/) (2024)
- [Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling](https://mlsanity.com/paper/2405.13796/) (2024)
- [Learning A Low-Level Vision Generalist via Visual Task Prompt](https://mlsanity.com/paper/2408.08601/) (2024)
- [Federated Prompt Learning for Weather Foundation Models on Devices](https://mlsanity.com/paper/2305.14244/) (2023)
- [UniFashion: A Unified Vision-Language Model for Multimodal Fashion Retrieval and Generation](https://mlsanity.com/paper/2408.11305/) (2024)
- [SEAL: A Framework for Systematic Evaluation of Real-World Super-Resolution](https://mlsanity.com/paper/2309.03020/) (2023)
- [PostCast: Generalizable Postprocessing for Precipitation Nowcasting via Unsupervised Blurriness Modeling](https://mlsanity.com/paper/2410.05805/) (2024)
- [Observation-Guided Meteorological Field Downscaling at Station Scale: A Benchmark and a New Method](https://mlsanity.com/paper/2401.11960/) (2024)
- [SRViT: Vision Transformers for Estimating Radar Reflectivity from Satellite Observations at Scale](https://mlsanity.com/paper/2406.16955/) (2024)
- [Benchmarking AI-based data assimilation to advance data-driven global weather forecasting](https://mlsanity.com/paper/2408.11438/) (2024)

## Built on by (most cited first, 12 of 12)

- [Deep Learning and Foundation Models for Weather Prediction: A Survey](https://mlsanity.com/paper/2501.06907/) (2025)
- [Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review](https://mlsanity.com/paper/2506.01364/) (2025)
- [WaveC2R: Wavelet-Driven Coarse-to-Refined Hierarchical Learning for Radar Retrieval](https://mlsanity.com/paper/2511.17558/) (2025)
- [Finetuning a Weather Foundation Model with Lightweight Decoders for Unseen Physical Processes](https://mlsanity.com/paper/2506.19088/) (2025)
- [Task-Adaptive Parameter-Efficient Fine-Tuning for Weather Foundation Models](https://mlsanity.com/paper/2509.22020/) (2025)
- [Omni-Weather: A Unified Multimodal Model for Weather Radar Understanding and Generation](https://mlsanity.com/paper/2512.21643/) (2025)
- [DAWP: A framework for global observation forecasting via Data Assimilation and Weather Prediction in satellite observation space](https://mlsanity.com/paper/2510.15978/) (2025)
- [SynWeather: Weather Observation Data Synthesis across Multiple Regions and Variables via a General Diffusion Transformer](https://mlsanity.com/paper/2511.08291/) (2025)
- [Safe Urban Traffic Control via Uncertainty-Aware Conformal Prediction and World-Model Reinforcement Learning](https://mlsanity.com/paper/2602.04821/) (2026)
- [Earth Science Foundation Models: From Perception to Reasoning and Discovery](https://mlsanity.com/paper/2605.12542/) (2026)
- [GLAIM: Learning Global and Local Adaptive Inter-Variable Dependency for Multivariate Time Series Imputation](https://mlsanity.com/paper/2608.02366/) (2026)
- [In-Context Inpainting for Time Series Forecasting](https://mlsanity.com/paper/2608.23855/) (2026)

## Abstract (arXiv)

The Earth's weather system encompasses intricate weather data modalities and diverse weather understanding tasks, which hold significant value to human life. Existing data-driven models focus on single weather understanding tasks (e.g., weather forecasting). Although these models have achieved promising results, they fail to tackle various complex tasks within a single and unified model. Moreover, the paradigm that relies on limited real observations for a single scenario hinders the model's performance upper bound. In response to these limitations, we draw inspiration from the in-context learning paradigm employed in state-of-the-art visual foundation models and large language models. In this paper, we introduce the first generalist weather foundation model (WeatherGFM), designed to address a wide spectrum of weather understanding tasks in a unified manner. More specifically, we initially unify the representation and definition of the diverse weather understanding tasks. Subsequently, we devised weather prompt formats to manage different weather data modalities, namely single, multiple, and temporal modalities. Finally, we adopt a visual prompting question-answering paradigm for the training of unified weather understanding tasks. Extensive experiments indicate that our WeatherGFM can effectively handle up to ten weather understanding tasks, including weather forecasting, super-resolution, weather image translation, and post-processing. Our method also showcases generalization ability on unseen tasks.