mlsanityData to 7 Oct 2026

Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformers

arXiv 2103.09360 ↗First posted 16 Mar 2021physics.ao-ph cs.AI cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

There is growing interest in data-driven weather prediction (DDWP), for example using convolutional neural networks such as U-NETs that are trained on data from models or reanalysis. Here, we propose 3 components to integrate with commonly used DDWP models in order to improve their physical consistency and forecast accuracy. These components are 1) a deep spatial transformer added to the latent space of the U-NETs to preserve a property called equivariance, which is related to correctly capturing rotations and scalings of features in spatio-temporal data, 2) a data-assimilation (DA) algorithm to ingest noisy observations and improve the initial conditions for next forecasts, and 3) a multi-time-step algorithm, which combines forecasts from DDWP models with different time steps through DA, improving the accuracy of forecasts at short intervals. To show the benefit/feasibility of each component, we use geopotential height at 500~hPa (Z500) from ERA5 reanalysis and examine the short-term forecast accuracy of specific setups of the DDWP framework. Results show that the equivariance-preserving networks (U-STNs) clearly outperform the U-NETs, for example improving the forecast skill by 45%. Using a sigma-point ensemble Kalman (SPEnKF) algorithm for DA and U-STN as the forward model, we show that stable, accurate DA cycles are achieved even with high observation noise. The DDWP+DA framework substantially benefits from large (O(1000)) ensembles that are inexpensively generated with the data-driven forward model in each DA cycle. The multi-time-step DDWP+DA framework also shows promises, e.g., it reduces the average error by factors of 2-3.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 23 of its 76 references that are in the map. First posted on arXiv 16 Mar 2021.

Builds on

23 of 76 references.

References · 23

  1. 2020WeatherBench: A benchmark dataset for data-driven weather forecastingcited by 272
  2. 2019Gauge Equivariant Convolutional Networks and the Icosahedral CNNcited by 262
  3. 2017Learning SO(3) Equivariant Representations with Spherical CNNscited by 244
  4. 2018Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networkscited by 155
  5. 2019On the Universality of Invariant Networkscited by 151
  6. 2020Improving data-driven global weather prediction using deep convolutional neural networks on a cubed spherecited by 95
  7. 2021Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction modelscited by 76
  8. 2020Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 modelcited by 72
  9. 2019A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problemscited by 59
  10. 2020Deep Learning for Post-Processing Ensemble Weather Forecastscited by 52
  11. 2020Combining data assimilation and machine learning to infer unresolved scale parametrisationcited by 40
  12. 2018Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Taskscited by 36
11 more
  1. 2019Analog forecasting of extreme-causing weather patterns using deep learningcited by 33
  2. 2019Achieving Conservation of Energy in Neural Network Emulators for Climate Modelingcited by 27
  3. 2019Physics-Guided Architecture (PGA) of Neural Networks for Quantifying Uncertainty in Lake Temperature Modelingcited by 26
  4. 2020Climbing down Charney's ladder: Machine Learning and the post-Dennard era of computational climate sciencecited by 18
  5. 2021Using Data Assimilation to Train a Hybrid Forecast System that Combines Machine-Learning and Knowledge-Based Componentscited by 18
  6. 2020Data-driven super-parameterization using deep learning: Experimentation with multi-scale Lorenz 96 systems and transfer-learningcited by 17
  7. 2018A test case for application of convolutional neural networks to spatio-temporal climate data: Re-identifying clustered weather patternscited by 15
  8. 2017Machine Learning, Deepest Learning: Statistical Data Assimilation Problemscited by 13
  9. 2021Reservoir Computing as a Tool for Climate Predictability Studiescited by 13
  10. 2020Analog ensemble data assimilation and a method for constructing analogs with variational autoencoderscited by 11
  11. 2020Designing Accurate Emulators for Scientific Processes using Calibration-Driven Deep Modelscited by 8

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 5; 2023: 3; 2024: 5; 2025: 1.

15 papers.

Sort:

  1. 2021Learning Dissipative Dynamics in Chaotic Systemscited by 41
  2. 2024Continuum Attention for Neural Operatorscited by 39
  3. 2023Foundation Models for Weather and Climate Data Understanding: A Comprehensive Surveycited by 32
  4. 2023W-MAE: Pre-trained weather model with masked autoencoder for multi-variable weather forecastingcited by 28
  5. 2023Using Sequences of Life-events to Predict Human Livescited by 27
  6. 2024Can AI weather models predict out-of-distribution gray swan tropical cyclones?cited by 26
  7. 2022Learning Spatiotemporal Chaos Using Next-Generation Reservoir Computingcited by 20
  8. 2022Statistical Deep Learning for Spatial and Spatio-Temporal Datacited by 15
  9. 2024Data Assimilation with Machine Learning Surrogate Models: A Case Study with FourCastNetcited by 10
  10. 2022Long-term stability and generalization of observationally-constrained stochastic data-driven models for geophysical turbulencecited by 8
  11. 2022A Multi-Scale Deep Learning Framework for Projecting Weather Extremescited by 7
  12. 2022A Novel Transformer Network with Shifted Window Cross-Attention for Spatiotemporal Weather Forecastingcited by 5
3 more
  1. 2025MODS: Multi-source Observations Conditional Diffusion Model for Meteorological State Downscalingcited by 3
  2. 2024Long-time accuracy of ensemble Kalman filters for chaotic and machine-learned dynamical systemscited by 1
  3. 2024Data Assimilation using ERA5, ASOS, and the U-STN model for Weather Forecasting over the UKcited by 0