# Generative Data Assimilation of Sparse Weather Station Observations at Kilometer Scales

arXiv: https://arxiv.org/abs/2406.16947 · HTML: https://mlsanity.com/paper/2406.16947/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 55 references that are in the map.
- First posted on arXiv 19 Jun 2024.

## Builds on (most cited first, 21 of 21)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [GraphCast: Learning skillful medium-range global weather forecasting](https://mlsanity.com/paper/2212.12794/) (2022)
- [Neural General Circulation Models for Weather and Climate](https://mlsanity.com/paper/2311.07222/) (2023)
- [A Variational Perspective on Solving Inverse Problems with Diffusion Models](https://mlsanity.com/paper/2305.04391/) (2023)
- [GenCast: Diffusion-based ensemble forecasting for medium-range weather](https://mlsanity.com/paper/2312.15796/) (2023)
- [Forecasting Global Weather with Graph Neural Networks](https://mlsanity.com/paper/2202.07575/) (2022)
- [Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast](https://mlsanity.com/paper/2211.02556/) (2022)
- [Score-based Data Assimilation](https://mlsanity.com/paper/2306.10574/) (2023)
- [Deep Learning for Day Forecasts from Sparse Observations](https://mlsanity.com/paper/2306.06079/) (2023)
- [DiffDA: a Diffusion Model for Weather-scale Data Assimilation](https://mlsanity.com/paper/2401.05932/) (2024)
- [Learning Diffusion Priors from Observations by Expectation Maximization](https://mlsanity.com/paper/2405.13712/) (2024)
- [Deep Generative Data Assimilation in Multimodal Setting](https://mlsanity.com/paper/2404.06665/) (2024)
- [Dynamical Tests of a Deep-Learning Weather Prediction Model](https://mlsanity.com/paper/2309.10867/) (2023)
- [Score-based Data Assimilation for a Two-Layer Quasi-Geostrophic Model](https://mlsanity.com/paper/2310.01853/) (2023)

## Built on by (most cited first, 27 of 27)

- [Kilometer-Scale Convection Allowing Model Emulation using Generative Diffusion Modeling](https://mlsanity.com/paper/2408.10958/) (2024)
- [Samudra: An AI Global Ocean Emulator for Climate](https://mlsanity.com/paper/2412.03795/) (2024)
- [Appa: Bending Weather Dynamics with Latent Diffusion Models for Global Data Assimilation](https://mlsanity.com/paper/2504.18720/) (2025)
- [Deep Learning and Foundation Models for Weather Prediction: A Survey](https://mlsanity.com/paper/2501.06907/) (2025)
- [Elucidated Rolling Diffusion Models for Probabilistic Forecasting of Complex Dynamics](https://mlsanity.com/paper/2506.20024/) (2025)
- [Decoupled Diffusion Sampling for Inverse Problems on Function Spaces](https://mlsanity.com/paper/2601.23280/) (2026)
- [Multimodal Atmospheric Super-Resolution With Deep Generative Models](https://mlsanity.com/paper/2506.22780/) (2025)
- [Optimization Benchmark for Diffusion Models on Dynamical Systems](https://mlsanity.com/paper/2510.19376/) (2025)
- [Align-DA: Align Score-based Atmospheric Data Assimilation with Multiple Preferences](https://mlsanity.com/paper/2505.22008/) (2025)
- [Benchmarking AI-based data assimilation to advance data-driven global weather forecasting](https://mlsanity.com/paper/2408.11438/) (2024)
- [How to use score-based diffusion in earth system science: A satellite nowcasting example](https://mlsanity.com/paper/2505.10432/) (2025)
- [Constructing Extreme Heatwave Storylines with Differentiable Climate Models](https://mlsanity.com/paper/2506.10660/) (2025)
- [Guided Unconditional and Conditional Generative Models for Super-Resolution and Inference of Quasi-Geostrophic Turbulence](https://mlsanity.com/paper/2507.00719/) (2025)
- [Global Attention with Linear Complexity for Exascale Generative Data Assimilation in Earth System Prediction](https://mlsanity.com/paper/2604.16590/) (2026)
- [Towards accurate extreme event likelihoods from diffusion model climate emulators](https://mlsanity.com/paper/2605.03802/) (2026)
- [Skillful high-resolution weather forecasting independent of physical models](https://mlsanity.com/paper/2605.28153/) (2026)
- [Control-Augmented Autoregressive Diffusion for Data Assimilation](https://mlsanity.com/paper/2510.06637/) (2025)
- [Sparse Local Implicit Image Function for sub-km Weather Downscaling](https://mlsanity.com/paper/2510.20228/) (2025)
- [Accurate and Efficient Hybrid-Ensemble Atmospheric Data Assimilation in Latent Space with Uncertainty Quantification](https://mlsanity.com/paper/2603.04395/) (2026)
- [Ensemble-based, large-eddy reconstruction of wind turbine inflow in a near-stationary atmospheric boundary layer through generative artificial intelligence](https://mlsanity.com/paper/2410.14024/) (2024)
- [PnP-DA: Towards Principled Plug-and-Play Integration of Variational Data Assimilation and Generative Models](https://mlsanity.com/paper/2508.00325/) (2025)
- [Omega: Operator-based Mixture Ensemble for Generative Assimilation](https://mlsanity.com/paper/2606.20920/) (2026)
- [Apeliotes: A Diffusion-Based Modeling Framework for km-scale Multi-Level Atmospheric Fields](https://mlsanity.com/paper/2607.17037/) (2026)
- [Generative data assimilation highlights fronts as key regulators of ocean energy cascade](https://mlsanity.com/paper/2608.14955/) (2026)
- [Steering Diffusion Priors with Sparse Observations for High-Resolution Temperature Downscaling](https://mlsanity.com/paper/2609.09247/) (2026)
- [Generative Atmospheric Super-Resolution from Heterogeneous In Situ Observations through Composable Interfaces](https://mlsanity.com/paper/2609.29027/) (2026)
- [Generative Priors Conditioned on Natural Language for Bayesian Inversion in PDEs](https://mlsanity.com/paper/2609.32941/) (2026)

## Abstract (arXiv)

Data assimilation of observational data into full atmospheric states is essential for weather forecast model initialization. Recently, methods for deep generative data assimilation have been proposed which allow for using new input data without retraining the model. They could also dramatically accelerate the costly data assimilation process used in operational regional weather models. Here, in a central US testbed, we demonstrate the viability of score-based data assimilation in the context of realistically complex km-scale weather. We train an unconditional diffusion model to generate snapshots of a state-of-the-art km-scale analysis product, the High Resolution Rapid Refresh. Then, using score-based data assimilation to incorporate sparse weather station data, the model produces maps of precipitation and surface winds. The generated fields display physically plausible structures, such as gust fronts, and sensitivity tests confirm learnt physics through multivariate relationships. Preliminary skill analysis shows the approach already outperforms a naive baseline of the High-Resolution Rapid Refresh system itself. By incorporating observations from 40 weather stations, 10% lower RMSEs on left-out stations are attained. Despite some lingering imperfections such as insufficiently disperse ensemble DA estimates, we find the results overall an encouraging proof of concept, and the first at km-scale. It is a ripe time to explore extensions that combine increasingly ambitious regional state generators with an increasing set of in situ, ground-based, and satellite remote sensing data streams.