Nipping the Butterfly Effect in the Bud: Self-Output Fine-Tuning for Autoregressive Weather Prediction
Not on the map. Why?
Abstract (arXiv)
Long-horizon weather forecasting is a fundamental challenge in atmospheric science, for which autoregressive Deep Learning Weather Prediction (DLWP) has emerged as the primary paradigm. Although the autoregressive pipeline is highly scalable and flexible, its prediction errors grow rapidly over long forecasting horizons. In this work, we study this error growth phenomenon from both theoretical and empirical perspectives. Our analysis reveals that the growth is driven by a feedback loop between output errors and input distribution shifts. Specifically, the autoregressive process amplifies small initial output errors, which progressively corrupt subsequent input distributions, echoing the butterfly effect in atmospheric science and ultimately deteriorating forecasting accuracy over longer horizons. Furthermore, we show that this distributional shift originates at the earliest stage of inference, with out-of-distribution signatures detectable as early as the first autoregressive step. To mitigate this issue, we propose Self-Output Fine-Tuning (SOFT), a plug-and-play strategy that leverages the model's own one-step predictions to calibrate the biased input distribution encountered at the first step. Extensive experiments demonstrate that, despite its simplicity, SOFT achieves state-of-the-art performance on long-horizon forecasting tasks and substantially reduces both prediction errors and distributional discrepancy. The success of SOFT highlights the importance of reexamining the fundamental pipeline of deep learning weather prediction, representing a critical pipeline advance for atmospheric science.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 9 of its 21 references that are in the map. First posted on arXiv 23 Jul 2026.
Builds on
9 of 21 references.
References · 9
- 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
- 2017Deep Image Priorcited by 1,097
- 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
- 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 748
- 2015Sequence Level Training with Recurrent Neural Networkscited by 568
- 2023FuXi: A cascade machine learning forecasting system for 15-day global weather forecastcited by 237
- 2022Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generationcited by 106
- 2024Neural Redshift: Random Networks are not Random Functionscited by 43
- 2024A Foundation Model for the Earth Systemcited by 23
Built on by
Not cited yet.