# CirT: Global Subseasonal-to-Seasonal Forecasting with Geometry-inspired Transformer

arXiv: https://arxiv.org/abs/2502.19750 · HTML: https://mlsanity.com/paper/2502.19750/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 54 references that are in the map.
- First posted on arXiv 27 Feb 2025.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Axial Attention in Multidimensional Transformers](https://mlsanity.com/paper/1912.12180/) (2019)
- [ClimaX: A foundation model for weather and climate](https://mlsanity.com/paper/2301.10343/) (2023)
- [WeatherBench: A benchmark dataset for data-driven weather forecasting](https://mlsanity.com/paper/2002.00469/) (2020)
- [GraphCast: Learning skillful medium-range global weather forecasting](https://mlsanity.com/paper/2212.12794/) (2022)
- [FuXi: A cascade machine learning forecasting system for 15-day global weather forecast](https://mlsanity.com/paper/2306.12873/) (2023)
- [WeatherBench 2: A benchmark for the next generation of data-driven global weather models](https://mlsanity.com/paper/2308.15560/) (2023)
- [Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere](https://mlsanity.com/paper/2306.03838/) (2023)
- [FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead](https://mlsanity.com/paper/2304.02948/) (2023)
- [Forecasting Global Weather with Graph Neural Networks](https://mlsanity.com/paper/2202.07575/) (2022)
- [Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere](https://mlsanity.com/paper/2003.11927/) (2020)
- [ClimODE: Climate and Weather Forecasting with Physics-informed Neural ODEs](https://mlsanity.com/paper/2404.10024/) (2024)
- [Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models](https://mlsanity.com/paper/2102.05107/) (2021)
- [ChaosBench: A Multi-Channel, Physics-Based Benchmark for Subseasonal-to-Seasonal Climate Prediction](https://mlsanity.com/paper/2402.00712/) (2024)
- [SEGNO: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biases](https://mlsanity.com/paper/2308.13212/) (2023)
- [SubseasonalClimateUSA: A Dataset for Subseasonal Forecasting and Benchmarking](https://mlsanity.com/paper/2109.10399/) (2021)
- [Adaptive Bias Correction for Improved Subseasonal Forecasting](https://mlsanity.com/paper/2209.10666/) (2022)
- [Modelling High-Order Social Relations for Item Recommendation](https://mlsanity.com/paper/2003.10149/) (2020)
- [Human Mobility Modeling During the COVID-19 Pandemic via Deep Graph Diffusion Infomax](https://mlsanity.com/paper/2212.05707/) (2022)
- [Learning and Dynamical Models for Sub-seasonal Climate Forecasting: Comparison and Collaboration](https://mlsanity.com/paper/2110.05196/) (2021)
- [CaFA: Global Weather Forecasting with Factorized Attention on Sphere](https://mlsanity.com/paper/2405.07395/) (2024)

## Built on by (most cited first, 14 of 14)

- [NeuralOM: Neural Ocean Model for Subseasonal-to-Seasonal Simulation](https://mlsanity.com/paper/2505.21020/) (2025)
- [Revitalizing Canonical Pre-Alignment for Irregular Multivariate Time Series Forecasting](https://mlsanity.com/paper/2508.01971/) (2025)
- [PEAR: Equal Area Weather Forecasting on the Sphere](https://mlsanity.com/paper/2505.17720/) (2025)
- [TianQuan-S2S: A Subseasonal-to-Seasonal Global Weather Model via Incorporate Climatology State](https://mlsanity.com/paper/2504.09940/) (2025)
- [A Survey of Cross-domain Graph Learning: Progress and Future Directions](https://mlsanity.com/paper/2503.11086/) (2025)
- [Physics-Informed Teleconnection-Aware Transformer for Global Subseasonal-to-Seasonal Forecasting](https://mlsanity.com/paper/2506.08049/) (2025)
- [PnP-Corrector: A Universal Correction Framework for Coupled Spatiotemporal Forecasting](https://mlsanity.com/paper/2605.08935/) (2026)
- [WeatherSyn: An Instruction Tuning MLLM For Weather Forecasting Report Generation](https://mlsanity.com/paper/2605.07522/) (2026)
- [CrossEarth-Gate: Fisher-Guided Adaptive Tuning Engine for Efficient Adaptation of Cross-Domain Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2511.20302/) (2025)
- [Skillful Subseasonal-to-Seasonal Forecasting of Extreme Events with a Multi-Sphere Coupled Probabilistic Model](https://mlsanity.com/paper/2512.12545/) (2025)
- [ParaFormer: A Generalized PageRank Graph Transformer for Graph Representation Learning](https://mlsanity.com/paper/2512.14619/) (2025)
- [Symmetric Composition of Anisotropic Operators for Global Subseasonal-to-Seasonal Climate Forecasting](https://mlsanity.com/paper/2602.15040/) (2026)
- [Fourier Geometric Wind Power Forecasting with Numerical Weather Prediction](https://mlsanity.com/paper/2607.17095/) (2026)
- [OceanFlow: Coherent Flow Matching for Probabilistic Ocean Modeling](https://mlsanity.com/paper/2610.15518/) (2026)

## Abstract (arXiv)

Accurate Subseasonal-to-Seasonal (S2S) climate forecasting is pivotal for decision-making including agriculture planning and disaster preparedness but is known to be challenging due to its chaotic nature. Although recent data-driven models have shown promising results, their performance is limited by inadequate consideration of geometric inductive biases. Usually, they treat the spherical weather data as planar images, resulting in an inaccurate representation of locations and spatial relations. In this work, we propose the geometric-inspired Circular Transformer (CirT) to model the cyclic characteristic of the graticule, consisting of two key designs: (1) Decomposing the weather data by latitude into circular patches that serve as input tokens to the Transformer; (2) Leveraging Fourier transform in self-attention to capture the global information and model the spatial periodicity. Extensive experiments on the Earth Reanalysis 5 (ERA5) reanalysis dataset demonstrate our model yields a significant improvement over the advanced data-driven models, including PanguWeather and GraphCast, as well as skillful ECMWF systems. Additionally, we empirically show the effectiveness of our model designs and high-quality prediction over spatial and temporal dimensions.