# Sonny: Breaking the Compute Wall in Medium-Range Weather Forecasting

arXiv: https://arxiv.org/abs/2603.21284 · HTML: https://mlsanity.com/paper/2603.21284/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 23 references that are in the map.
- First posted on arXiv 22 Mar 2026.

## Builds on (most cited first, 12 of 12)

- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere](https://mlsanity.com/paper/2306.03838/) (2023)
- [Scaling transformer neural networks for skillful and reliable medium-range weather forecasting](https://mlsanity.com/paper/2312.03876/) (2023)
- [FourCastNet 3: A geometric approach to probabilistic machine-learning weather forecasting at scale](https://mlsanity.com/paper/2507.12144/) (2025)
- [FengWu-GHR: Learning the Kilometer-scale Medium-range Global Weather Forecasting](https://mlsanity.com/paper/2402.00059/) (2024)
- [KARINA: An Efficient Deep Learning Model for Global Weather Forecast](https://mlsanity.com/paper/2403.10555/) (2024)
- [FastNet: Improving the physical consistency of machine-learning weather prediction models through loss function design](https://mlsanity.com/paper/2509.17601/) (2025)
- [Democracy of AI Numerical Weather Models: An Example of Global Forecasting with FourCastNetv2 Made by a University Research Lab Using GPU](https://mlsanity.com/paper/2504.17028/) (2025)
- [Step by Step Network](https://mlsanity.com/paper/2511.14329/) (2025)
- [EMFormer: Efficient Multi-Scale Transformer for Accumulative Context Weather Forecasting](https://mlsanity.com/paper/2602.01194/) (2026)

## Built on by (most cited first, 2 of 2)

- [Beyond Linear Superposition: Discovering Climate Features in AI Weather Models with KAN-SAE](https://mlsanity.com/paper/2605.17493/) (2026)
- [Rescene: band-limited stochastic forcing turns a frozen neural weather operator into a climate emulator](https://mlsanity.com/paper/2608.09971/) (2026)

## Abstract (arXiv)

Weather forecasting is a fundamental problem for protecting lives and infrastructure from high-impact atmospheric events. Recently, data-driven weather forecasting methods based on deep learning have demonstrated strong performance, often reaching accuracy levels competitive with operational numerical systems. However, many existing models rely on large-scale training regimes and compute-intensive architectures, which raises the practical barrier for academic groups with limited compute resources. Here we introduce Sonny, an efficient hierarchical transformer that achieves competitive medium-range forecasting performance while remaining feasible within reasonable compute budgets. At the core of Sonny is a two-stage StepsNet design: a narrow slow path first models large-scale atmospheric dynamics, and a subsequent full-width fast path integrates thermodynamic interactions. To stabilize medium-range rollout without an additional fine-tuning stage, we apply exponential moving average (EMA) during training. On WeatherBench2, Sonny yields robust medium-range forecast skill, remains competitive with operational baselines, and demonstrates clear advantages over FastNet, particularly at extended tropical lead times. In practice, Sonny can be trained to convergence on a single NVIDIA A40 GPU in approximately 5.5 days.