# Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast

arXiv: https://arxiv.org/abs/2211.02556 · HTML: https://mlsanity.com/paper/2211.02556/

## Key facts

- Cited by 166 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 59 references that are in the map.
- First posted on arXiv 3 Nov 2022.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Skillful Precipitation Nowcasting using Deep Generative Models of Radar](https://mlsanity.com/paper/2104.00954/) (2021)
- [WeatherBench: A benchmark dataset for data-driven weather forecasting](https://mlsanity.com/paper/2002.00469/) (2020)
- [Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model](https://mlsanity.com/paper/1706.03458/) (2017)
- [Forecasting Global Weather with Graph Neural Networks](https://mlsanity.com/paper/2202.07575/) (2022)
- [MetNet: A Neural Weather Model for Precipitation Forecasting](https://mlsanity.com/paper/2003.12140/) (2020)
- [Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models](https://mlsanity.com/paper/2102.05107/) (2021)
- [Machine Learning for Precipitation Nowcasting from Radar Images](https://mlsanity.com/paper/1912.12132/) (2019)
- [SwinVRNN: A Data-Driven Ensemble Forecasting Model via Learned Distribution Perturbation](https://mlsanity.com/paper/2205.13158/) (2022)
- [Precipitation Nowcasting with Satellite Imagery](https://mlsanity.com/paper/1905.09932/) (2019)

## Built on by (most cited first, 30 of 166)

- [ClimaX: A foundation model for weather and climate](https://mlsanity.com/paper/2301.10343/) (2023)
- [GraphCast: Learning skillful medium-range global weather forecasting](https://mlsanity.com/paper/2212.12794/) (2022)
- [FuXi: A cascade machine learning forecasting system for 15-day global weather forecast](https://mlsanity.com/paper/2306.12873/) (2023)
- [Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere](https://mlsanity.com/paper/2306.03838/) (2023)
- [GenCast: Diffusion-based ensemble forecasting for medium-range weather](https://mlsanity.com/paper/2312.15796/) (2023)
- [FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead](https://mlsanity.com/paper/2304.02948/) (2023)
- [Mamba-ND: Selective State Space Modeling for Multi-Dimensional Data](https://mlsanity.com/paper/2402.05892/) (2024)
- [Deep Learning for Day Forecasts from Sparse Observations](https://mlsanity.com/paper/2306.06079/) (2023)
- [3D and 4D World Modeling: A Survey](https://mlsanity.com/paper/2509.07996/) (2025)
- [FourCastNet 3: A geometric approach to probabilistic machine-learning weather forecasting at scale](https://mlsanity.com/paper/2507.12144/) (2025)
- [Towards Stability of Autoregressive Neural Operators](https://mlsanity.com/paper/2306.10619/) (2023)
- [Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures](https://mlsanity.com/paper/2311.00636/) (2023)
- [ClimateLearn: Benchmarking Machine Learning for Weather and Climate Modeling](https://mlsanity.com/paper/2307.01909/) (2023)
- [ChaosBench: A Multi-Channel, Physics-Based Benchmark for Subseasonal-to-Seasonal Climate Prediction](https://mlsanity.com/paper/2402.00712/) (2024)
- [Geometric Clifford Algebra Networks](https://mlsanity.com/paper/2302.06594/) (2023)
- [On the Foundations of Earth and Climate Foundation Models](https://mlsanity.com/paper/2405.04285/) (2024)
- [Prithvi WxC: Foundation Model for Weather and Climate](https://mlsanity.com/paper/2409.13598/) (2024)
- [W-MAE: Pre-trained weather model with masked autoencoder for multi-variable weather forecasting](https://mlsanity.com/paper/2304.08754/) (2023)
- [AI-GOMS: Large AI-Driven Global Ocean Modeling System](https://mlsanity.com/paper/2308.03152/) (2023)
- [Universal Physics Transformers: A Framework For Efficiently Scaling Neural Operators](https://mlsanity.com/paper/2402.12365/) (2024)
- [Generative Data Assimilation of Sparse Weather Station Observations at Kilometer Scales](https://mlsanity.com/paper/2406.16947/) (2024)
- [ClimateSet: A Large-Scale Climate Model Dataset for Machine Learning](https://mlsanity.com/paper/2311.03721/) (2023)
- [Generative Diffusion-based Downscaling for Climate](https://mlsanity.com/paper/2404.17752/) (2024)
- [Lie Point Symmetry and Physics Informed Networks](https://mlsanity.com/paper/2311.04293/) (2023)
- [Precipitation nowcasting with generative diffusion models](https://mlsanity.com/paper/2308.06733/) (2023)
- [Diffusion Models for High-Resolution Solar Forecasts](https://mlsanity.com/paper/2302.00170/) (2023)
- [ORBIT: Oak Ridge Base Foundation Model for Earth System Predictability](https://mlsanity.com/paper/2404.14712/) (2024)
- [Analyzing and Exploring Training Recipes for Large-Scale Transformer-Based Weather Prediction](https://mlsanity.com/paper/2404.19630/) (2024)
- [Environmental Sensor Placement with Convolutional Gaussian Neural Processes](https://mlsanity.com/paper/2211.10381/) (2022)
- [A Survey of Generative Techniques for Spatial-Temporal Data Mining](https://mlsanity.com/paper/2405.09592/) (2024)

## Abstract (arXiv)

In this paper, we present Pangu-Weather, a deep learning based system for fast and accurate global weather forecast. For this purpose, we establish a data-driven environment by downloading 43 years of hourly global weather data from the 5th generation of ECMWF reanalysis (ERA5) data and train a few deep neural networks with about 256 million parameters in total. The spatial resolution of forecast is 0.25^circ×0.25^circ, comparable to the ECMWF Integrated Forecast Systems (IFS). More importantly, for the first time, an AI-based method outperforms state-of-the-art numerical weather prediction (NWP) methods in terms of accuracy (latitude-weighted RMSE and ACC) of all factors (e.g., geopotential, specific humidity, wind speed, temperature, etc.) and in all time ranges (from one hour to one week). There are two key strategies to improve the prediction accuracy: (i) designing a 3D Earth Specific Transformer (3DEST) architecture that formulates the height (pressure level) information into cubic data, and (ii) applying a hierarchical temporal aggregation algorithm to alleviate cumulative forecast errors. In deterministic forecast, Pangu-Weather shows great advantages for short to medium-range forecast (i.e., forecast time ranges from one hour to one week). Pangu-Weather supports a wide range of downstream forecast scenarios, including extreme weather forecast (e.g., tropical cyclone tracking) and large-member ensemble forecast in real-time. Pangu-Weather not only ends the debate on whether AI-based methods can surpass conventional NWP methods, but also reveals novel directions for improving deep learning weather forecast systems.