mlsanityData to 14 Oct 2026

PhysMetrics.Weather: An Evaluation Framework for Physical Consistency in ML Weather Models

arXiv 2606.10642 ↗First posted 9 Jun 2026cs.LG physics.ao-phcited by 5

Not on the map. Why?

Abstract (arXiv)

Machine learning weather prediction (MLWP) models have achieved impressive forecasting performance at a small fraction of the computational costs required for traditional physics-based methods. However, they are primarily (1) data-driven and (2) evaluated using pixel-wide error metrics (e.g., RMSE), so there are no guarantees that their forecasts are consistent with known physical laws. We introduce PhysMetrics.Weather, an evaluation framework that assesses the physical realism of MLWP models across three types of metrics: conservation, spectral, and dynamical. By quantifying physical realism, this tool guides the development of physics-informed architectures and helps evaluate whether MLWP models are reliable for operational use. Our framework is available on Github at https://github.com/Emmakast/PhysMetrics.Weather.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 13 of its 39 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

13 of 39 references.

References · 13

  1. 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 776
  2. 2020WeatherBench: A benchmark dataset for data-driven weather forecastingcited by 275
  3. 2022GraphCast: Learning skillful medium-range global weather forecastingcited by 267
  4. 2023Neural General Circulation Models for Weather and Climatecited by 239
  5. 2023WeatherBench 2: A benchmark for the next generation of data-driven global weather modelscited by 236
  6. 2023GenCast: Diffusion-based ensemble forecasting for medium-range weathercited by 177
  7. 2022Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecastcited by 166
  8. 2024ClimODE: Climate and Weather Forecasting with Physics-informed Neural ODEscited by 82
  9. 2024ChaosBench: A Multi-Channel, Physics-Based Benchmark for Subseasonal-to-Seasonal Climate Predictioncited by 43
  10. 2025Fixing the Double Penalty in Data-Driven Weather Forecasting Through a Modified Spherical Harmonic Loss Functioncited by 28
  11. 2024FuXi Weather: A data-to-forecast machine learning system for global weathercited by 21
  12. 2025FastNet: Improving the physical consistency of machine-learning weather prediction models through loss function designcited by 5
1 more
  1. 2025Assessing the Geographic Generalization and Physical Consistency of Generative Models for Climate Downscalingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Toward Mechanistic Interpretability of an AI Foundation Model Fine-Tuned for Atmospheric Chemistrycited by 1
  2. 2026Stochastically Perturbed Weights: Ensembles from Deterministic Machine-Learning Weather Modelscited by 0
  3. 2026Every Fixed Metric Has a Blind Spot: A Learned Atmospheric Critic for Scoring Forecast Realismcited by 0
  4. 2026Evaluating Dynamical Fidelity through Predictive Structure in Physical Representationscited by 0
  5. 2026Strategic Governance of AI Models in Earth Sciencecited by 0