# Long-Range Distillation: Distilling 10,000 Years of Simulated Climate into Long Timestep AI Weather Models

arXiv: https://arxiv.org/abs/2512.22814 · HTML: https://mlsanity.com/paper/2512.22814/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 44 references that are in the map.
- First posted on arXiv 28 Dec 2025.

## Builds on (most cited first, 19 of 19)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [FuXi: A cascade machine learning forecasting system for 15-day global weather forecast](https://mlsanity.com/paper/2306.12873/) (2023)
- [FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead](https://mlsanity.com/paper/2304.02948/) (2023)
- [Forecasting Global Weather with Graph Neural Networks](https://mlsanity.com/paper/2202.07575/) (2022)
- [MetNet: A Neural Weather Model for Precipitation Forecasting](https://mlsanity.com/paper/2003.12140/) (2020)
- [Scaling transformer neural networks for skillful and reliable medium-range weather forecasting](https://mlsanity.com/paper/2312.03876/) (2023)
- [Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere](https://mlsanity.com/paper/2003.11927/) (2020)
- [Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models](https://mlsanity.com/paper/2102.05107/) (2021)
- [Skillful joint probabilistic weather forecasting from marginals](https://mlsanity.com/paper/2506.10772/) (2025)
- [FourCastNet 3: A geometric approach to probabilistic machine-learning weather forecasting at scale](https://mlsanity.com/paper/2507.12144/) (2025)
- [ACE2: Accurately learning subseasonal to decadal atmospheric variability and forced responses](https://mlsanity.com/paper/2411.11268/) (2024)
- [Samudra: An AI Global Ocean Emulator for Climate](https://mlsanity.com/paper/2412.03795/) (2024)
- [A Practical Probabilistic Benchmark for AI Weather Models](https://mlsanity.com/paper/2401.15305/) (2024)
- [SamudrACE: Fast and Accurate Coupled Climate Modeling with 3D Ocean and Atmosphere Emulators](https://mlsanity.com/paper/2509.12490/) (2025)
- [An ensemble of data-driven weather prediction models for operational sub-seasonal forecasting](https://mlsanity.com/paper/2403.15598/) (2024)

## Built on by (most cited first, 2 of 2)

- [Towards accurate extreme event likelihoods from diffusion model climate emulators](https://mlsanity.com/paper/2605.03802/) (2026)
- [SimCast-S2S: A Computationally Efficient Diffusion Model for Subseasonal Precipitation Forecasting](https://mlsanity.com/paper/2608.26594/) (2026)

## Abstract (arXiv)

Accurate long-range weather forecasting remains a major challenge for AI models, both because errors accumulate over autoregressive rollouts and because reanalysis training datasets provide limited sampling of the slow modes of variability underpinning predictability. Most AI and all numerical weather prediction models are autoregressive, repeatedly applying short-lead forecasts to reach subseasonal-to-seasonal (S2S) or seasonal lead times, often causing instability and miscalibration. Long-timestep probabilistic models that generate long-range forecasts in a single step offer an alternative, but training on the 40-year reanalysis record leads to overfitting, suggesting orders of magnitude more data are required. We introduce long-range distillation, whereby a long-timestep probabilistic ``student'' model is trained to forecast directly at long-range using a huge synthetic dataset generated by an autoregressive ``teacher'' model. Using the Deep Learning Earth System Model (DLESyM) as teacher, we generate over 10,000 years of simulated climate to train student models across a range of timescales. In perfect-model experiments, the distilled models outperform climatology and approach their teacher's skill while replacing hundreds of autoregressive steps with a single timestep. In the real world, both a diffusion model student fine-tuned on ERA5 and a linear inverse model student with mixed DLESyM-ERA5 training achieve S2S forecast skill competitive with the ECMWF IFS ensemble, with synthetic pre-training providing improvements beyond ERA5-only training. This demonstrates the benefit of synthetic pre-training for both deep learning and classical data-driven methods, and represents the first demonstration that real-world skill can be improved by training on AI-generated synthetic data in volumes far exceeding the observational record.