# ChaosBench: A Multi-Channel, Physics-Based Benchmark for Subseasonal-to-Seasonal Climate Prediction

arXiv: https://arxiv.org/abs/2402.00712 · HTML: https://mlsanity.com/paper/2402.00712/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 92 references that are in the map.
- First posted on arXiv 1 Feb 2024.

## Builds on (most cited first, 20 of 20)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [ClimaX: A foundation model for weather and climate](https://mlsanity.com/paper/2301.10343/) (2023)
- [WeatherBench: A benchmark dataset for data-driven weather forecasting](https://mlsanity.com/paper/2002.00469/) (2020)
- [GraphCast: Learning skillful medium-range global weather forecasting](https://mlsanity.com/paper/2212.12794/) (2022)
- [WeatherBench 2: A benchmark for the next generation of data-driven global weather models](https://mlsanity.com/paper/2308.15560/) (2023)
- [Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast](https://mlsanity.com/paper/2211.02556/) (2022)
- [MetNet: A Neural Weather Model for Precipitation Forecasting](https://mlsanity.com/paper/2003.12140/) (2020)
- [Scaling transformer neural networks for skillful and reliable medium-range weather forecasting](https://mlsanity.com/paper/2312.03876/) (2023)
- [Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models](https://mlsanity.com/paper/2102.05107/) (2021)
- [ClimateSet: A Large-Scale Climate Model Dataset for Machine Learning](https://mlsanity.com/paper/2311.03721/) (2023)
- [SubseasonalClimateUSA: A Dataset for Subseasonal Forecasting and Benchmarking](https://mlsanity.com/paper/2109.10399/) (2021)
- [AI Foundation Models for Weather and Climate: Applications, Design, and Implementation](https://mlsanity.com/paper/2309.10808/) (2023)
- [Adaptive Bias Correction for Improved Subseasonal Forecasting](https://mlsanity.com/paper/2209.10666/) (2022)
- [Improving Subseasonal Forecasting in the Western U.S. with Machine Learning](https://mlsanity.com/paper/1809.07394/) (2018)
- [Aboveground carbon biomass estimate with Physics-informed deep network](https://mlsanity.com/paper/2210.13752/) (2022)

## Built on by (most cited first, 30 of 43)

- [Deep Generative Data Assimilation in Multimodal Setting](https://mlsanity.com/paper/2404.06665/) (2024)
- [RealPDEBench: A Benchmark for Complex Physical Systems with Real-World Data](https://mlsanity.com/paper/2601.01829/) (2026)
- [CirT: Global Subseasonal-to-Seasonal Forecasting with Geometry-inspired Transformer](https://mlsanity.com/paper/2502.19750/) (2025)
- [OmniCast: A Masked Latent Diffusion Model for Weather Forecasting Across Time Scales](https://mlsanity.com/paper/2510.18707/) (2025)
- [PhyDA: Physics-Guided Diffusion Models for Data Assimilation in Atmospheric Systems](https://mlsanity.com/paper/2505.12882/) (2025)
- [CausalDynamics: A large-scale benchmark for structural discovery of dynamical causal models](https://mlsanity.com/paper/2505.16620/) (2025)
- [Generative emulation of chaotic dynamics with coherent prior](https://mlsanity.com/paper/2504.14264/) (2025)
- [Utilizing Strategic Pre-training to Reduce Overfitting: Baguan -- A Pre-trained Weather Forecasting Model](https://mlsanity.com/paper/2505.13873/) (2025)
- [Self-Supervised Evolution Operator Learning for High-Dimensional Dynamical Systems](https://mlsanity.com/paper/2505.18671/) (2025)
- [Joint Parameter and Parameterization Inference with Uncertainty Quantification through Differentiable Programming](https://mlsanity.com/paper/2403.02215/) (2024)
- [PhysMetrics.Weather: An Evaluation Framework for Physical Consistency in ML Weather Models](https://mlsanity.com/paper/2606.10642/) (2026)
- [Causal Spatio-Temporal Prediction: An Effective and Efficient Multi-Modal Approach](https://mlsanity.com/paper/2505.17637/) (2025)
- [DiffScale: Continuous Downscaling and Bias Correction of Subseasonal Wind Speed Forecasts using Diffusion Models](https://mlsanity.com/paper/2503.23893/) (2025)
- [TianQuan-S2S: A Subseasonal-to-Seasonal Global Weather Model via Incorporate Climatology State](https://mlsanity.com/paper/2504.09940/) (2025)
- [Decision-oriented benchmarking to transform AI weather forecast access: Application to the Indian monsoon](https://mlsanity.com/paper/2602.03767/) (2026)
- [Deep Koopman operator framework for causal discovery in nonlinear dynamical systems](https://mlsanity.com/paper/2505.14828/) (2025)
- [Physics-Informed Teleconnection-Aware Transformer for Global Subseasonal-to-Seasonal Forecasting](https://mlsanity.com/paper/2506.08049/) (2025)
- [LangPrecip: Language-Aware Multimodal Precipitation Nowcasting](https://mlsanity.com/paper/2512.22317/) (2025)
- [Neural Stochastic Processes for Satellite Precipitation Refinement](https://mlsanity.com/paper/2604.10414/) (2026)
- [Advancing Subseasonal Forecasting with Machine Learning](https://mlsanity.com/paper/2604.16238/) (2026)
- [ConDiff: A Challenging Dataset for Neural Solvers of Partial Differential Equations](https://mlsanity.com/paper/2406.04709/) (2024)
- [ClimDetect: A Benchmark Dataset for Climate Change Detection and Attribution](https://mlsanity.com/paper/2408.15993/) (2024)
- [LatticeVision: Image to Image Networks for Modeling Non-Stationary Spatial Data](https://mlsanity.com/paper/2505.09803/) (2025)
- [IndiaWeatherBench: A Dataset and Benchmark for Data-Driven Regional Weather Forecasting over India](https://mlsanity.com/paper/2509.00653/) (2025)
- [DAMBench: A Multi-Modal Benchmark for Deep Learning-based Atmospheric Data Assimilation](https://mlsanity.com/paper/2511.01468/) (2025)
- [Accurate and Efficient Hybrid-Ensemble Atmospheric Data Assimilation in Latent Space with Uncertainty Quantification](https://mlsanity.com/paper/2603.04395/) (2026)
- [WeatherSyn: An Instruction Tuning MLLM For Weather Forecasting Report Generation](https://mlsanity.com/paper/2605.07522/) (2026)
- [GrINd: Grid Interpolation Network for Scattered Observations](https://mlsanity.com/paper/2403.19570/) (2024)
- [KunPeng: A Global Ocean Environmental Model](https://mlsanity.com/paper/2504.04766/) (2025)
- [Setting the Standard: Recommended Practices for Data Preprocessing in Data-Driven Climate Prediction](https://mlsanity.com/paper/2508.07062/) (2025)

## Abstract (arXiv)

Accurate prediction of climate in the subseasonal-to-seasonal scale is crucial for disaster preparedness and robust decision making amidst climate change. Yet, forecasting beyond the weather timescale is challenging because it deals with problems other than initial condition, including boundary interaction, butterfly effect, and our inherent lack of physical understanding. At present, existing benchmarks tend to have shorter forecasting range of up-to 15 days, do not include a wide range of operational baselines, and lack physics-based constraints for explainability. Thus, we propose ChaosBench, a challenging benchmark to extend the predictability range of data-driven weather emulators to S2S timescale. First, ChaosBench is comprised of variables beyond the typical surface-atmospheric ERA5 to also include ocean, ice, and land reanalysis products that span over 45 years to allow for full Earth system emulation that respects boundary conditions. We also propose physics-based, in addition to deterministic and probabilistic metrics, to ensure a physically-consistent ensemble that accounts for butterfly effect. Furthermore, we evaluate on a diverse set of physics-based forecasts from four national weather agencies as baselines to our data-driven counterpart such as ViT/ClimaX, PanguWeather, GraphCast, and FourCastNetV2. Overall, we find methods originally developed for weather-scale applications fail on S2S task: their performance simply collapse to an unskilled climatology. Nonetheless, we outline and demonstrate several strategies that can extend the predictability range of existing weather emulators, including the use of ensembles, robust control of error propagation, and the use of physics-informed models. Our benchmark, datasets, and instructions are available at https://leap-stc.github.io/ChaosBench.