# Learning to forecast diagnostic parameters using pre-trained weather embedding

arXiv: https://arxiv.org/abs/2312.00290 · HTML: https://mlsanity.com/paper/2312.00290/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 13 references that are in the map.
- First posted on arXiv 1 Dec 2023.

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [ClimaX: A foundation model for weather and climate](https://mlsanity.com/paper/2301.10343/) (2023)
- [GraphCast: Learning skillful medium-range global weather forecasting](https://mlsanity.com/paper/2212.12794/) (2022)
- [Understanding The Robustness in Vision Transformers](https://mlsanity.com/paper/2204.12451/) (2022)
- [How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding](https://mlsanity.com/paper/2303.04245/) (2023)
- [W-MAE: Pre-trained weather model with masked autoencoder for multi-variable weather forecasting](https://mlsanity.com/paper/2304.08754/) (2023)
- [Dynamical Tests of a Deep-Learning Weather Prediction Model](https://mlsanity.com/paper/2309.10867/) (2023)
- [Verification against in-situ observations for Data-Driven Weather Prediction](https://mlsanity.com/paper/2305.00048/) (2023)

## Built on by (most cited first, 3 of 3)

- [Huge Ensembles Part I: Design of Ensemble Weather Forecasts using Spherical Fourier Neural Operators](https://mlsanity.com/paper/2408.03100/) (2024)
- [Examining Fast Radiatively Driven Responses Using Machine-Learning Weather Emulators](https://mlsanity.com/paper/2602.16090/) (2026)
- [AGCD: Agent-Guided Cross-Modal Decoding for Weather Forecasting](https://mlsanity.com/paper/2603.15260/) (2026)

## Abstract (arXiv)

Data-driven weather prediction (DDWP) models are increasingly becoming popular for weather forecasting. However, while operational weather forecasts predict a wide variety of weather variables, DDWPs currently forecast a specific set of key prognostic variables. Non-prognostic ("diagnostic") variables are sometimes modeled separately as dependent variables of the prognostic variables (c.f. FourCastNet), or by including the diagnostic variable as a target in the DDWP. However, the cost of training and deploying bespoke models for each diagnostic variable can increase dramatically with more diagnostic variables, and limit the operational use of such models. Likewise, retraining an entire DDWP each time a new diagnostic variable is added is also cost-prohibitive. We present an two-stage approach that allows new diagnostic variables to be added to an end-to-end DDWP model without the expensive retraining. In the first stage, we train an autoencoder that learns to embed prognostic variables into a latent space. In the second stage, the autoencoder is frozen and "downstream" models are trained to predict diagnostic variables using only the latent representations of prognostic variables as input. Our experiments indicate that models trained using the two-stage approach offer accuracy comparable to training bespoke models, while leading to significant reduction in resource utilization during training and inference. This approach allows for new "downstream" models to be developed as needed, without affecting existing models and thus reducing the friction in operationalizing new models.