Stable Attention Response for Reliable Precipitation Nowcasting
Not on the map. Why?
Abstract (arXiv)
Precipitation nowcasting remains challenging due to the highly localized, rapidly evolving, and heterogeneous nature of atmospheric dynamics. Although recent methods increasingly adopt attention-based architectures in both unimodal and multimodal settings, they mainly emphasize stronger representation learning and prediction capacity, while paying less attention to the stability of attention responses across samples. In this work, we show that cross-sample instability of attention-response energy is an important and previously underexplored source of forecasting unreliability. Empirically, inaccurate forecasts are associated with larger attention-response energy variance across heads and layers. Theoretically, we show that cross-sample variability can propagate through self-attention, and enlarge a lower bound on prediction error. Based on this insight, we propose HARECast, a Head-wise Attention Response Energy-regulated framework for precipitation nowcasting. HARECast explicitly models head-wise attention-response energy and stabilizes it through a group-wise regularization objective that reduces cross-sample fluctuations. The proposed formulation is generic and applicable to both unimodal and multimodal nowcasting architectures. We instantiate HARECast in a standard forecasting pipeline with reconstruction branches and a diffusion-based predictor, and evaluate it on commonly used benchmarks--SEVIR and MeteoNet. Experimental results demonstrate that HARECast achieves state-of-the-art performance.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 14 of its 35 references that are in the map. First posted on arXiv 13 May 2026.
Builds on
14 of 35 references.
References · 14
- 2017Attention Is All You Needcited by 60,361
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
- 2019Are Sixteen Heads Really Better than One?cited by 816
- 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 776
- 2020Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Predictioncited by 195
- 2022SimVP: Simpler yet Better Video Predictioncited by 184
- 2017Deep Learning for Precipitation Nowcasting: A Benchmark and A New Modelcited by 182
- 2022Earthformer: Exploring Space-Time Transformers for Earth System Forecastingcited by 152
- 2023DiffCast: A Unified Framework via Residual Diffusion for Precipitation Nowcastingcited by 61
- 2024CasCast: Skillful High-resolution Precipitation Nowcasting via Cascaded Modellingcited by 56
Built on by
Citing papers per year: 2026: 2.
2 papers.