# AA-TransUNet: Attention Augmented TransUNet For Nowcasting Tasks

arXiv: https://arxiv.org/abs/2202.04996 · HTML: https://mlsanity.com/paper/2202.04996/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 10 Feb 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [DeepViT: Towards Deeper Vision Transformer](https://mlsanity.com/paper/2103.11886/) (2021)
- [Stochastic Video Generation with a Learned Prior](https://mlsanity.com/paper/1802.07687/) (2018)
- [Investigating the Limitations of Transformers with Simple Arithmetic Tasks](https://mlsanity.com/paper/2102.13019/) (2021)
- [Vision Transformer for Small-Size Datasets](https://mlsanity.com/paper/2112.13492/) (2021)
- [SmaAt-UNet: Precipitation Nowcasting using a Small Attention-UNet Architecture](https://mlsanity.com/paper/2007.04417/) (2020)
- [Machine Learning for Precipitation Nowcasting from Radar Images](https://mlsanity.com/paper/1912.12132/) (2019)
- [Towards Learning Convolutions from Scratch](https://mlsanity.com/paper/2007.13657/) (2020)
- [Broad-UNet: Multi-scale feature learning for nowcasting tasks](https://mlsanity.com/paper/2102.06442/) (2021)
- [Deep Graph Convolutional Networks for Wind Speed Prediction](https://mlsanity.com/paper/2101.10041/) (2021)
- [Wind speed prediction using multidimensional convolutional neural networks](https://mlsanity.com/paper/2007.12567/) (2020)
- [Symbolic regression for scientific discovery: an application to wind speed forecasting](https://mlsanity.com/paper/2102.10570/) (2021)
- [Deep coastal sea elements forecasting using U-Net based models](https://mlsanity.com/paper/2011.03303/) (2020)
- [Multistream Graph Attention Networks for Wind Speed Forecasting](https://mlsanity.com/paper/2108.07063/) (2021)
- [TENT: Tensorized Encoder Transformer for Temperature Forecasting](https://mlsanity.com/paper/2106.14742/) (2021)
- [Cloud Cover Nowcasting with Deep Learning](https://mlsanity.com/paper/2009.11577/) (2020)

## Built on by (most cited first, 22 of 22)

- [DA-TransUNet: Integrating Spatial and Channel Dual Attention with Transformer U-Net for Medical Image Segmentation](https://mlsanity.com/paper/2310.12570/) (2023)
- [Continual Hippocampus Segmentation with Transformers](https://mlsanity.com/paper/2204.08043/) (2022)
- [GA-SmaAt-GNet: Generative Adversarial Small Attention GNet for Extreme Precipitation Nowcasting](https://mlsanity.com/paper/2401.09881/) (2024)
- [Graph Dual-stream Convolutional Attention Fusion for Precipitation Nowcasting](https://mlsanity.com/paper/2401.07958/) (2024)
- [SAR-UNet: Small Attention Residual UNet for Explainable Nowcasting Tasks](https://mlsanity.com/paper/2303.06663/) (2023)
- [Extreme Precipitation Nowcasting using Transformer-based Generative Models](https://mlsanity.com/paper/2403.03929/) (2024)
- [WaveC2R: Wavelet-Driven Coarse-to-Refined Hierarchical Learning for Radar Retrieval](https://mlsanity.com/paper/2511.17558/) (2025)
- [WF-UNet: Weather Fusion UNet for Precipitation Nowcasting](https://mlsanity.com/paper/2302.04102/) (2023)
- [SSA-UNet: Advanced Precipitation Nowcasting via Channel Shuffling](https://mlsanity.com/paper/2504.18309/) (2025)
- [Probabilistic Precipitation Nowcasting with Rectified Flow Transformers](https://mlsanity.com/paper/2605.31204/) (2026)
- [TransCORALNet: A Two-Stream Transformer CORAL Networks for Supply Chain Credit Assessment Cold Start](https://mlsanity.com/paper/2311.18749/) (2023)
- [A Mutual Inclusion Mechanism for Precise Boundary Segmentation in Medical Images](https://mlsanity.com/paper/2404.08201/) (2024)
- [J-CaPA : Joint Channel and Pyramid Attention Improves Medical Image Segmentation](https://mlsanity.com/paper/2411.16568/) (2024)
- [Self-supervised Spatial-Temporal Learner for Precipitation Nowcasting](https://mlsanity.com/paper/2412.15917/) (2024)
- [MFC-RFNet: A Multi-scale Guided Rectified Flow Network for Radar Sequence Prediction](https://mlsanity.com/paper/2601.03633/) (2026)
- [MAD-SmaAt-GNet: A Multimodal Advection-Guided Neural Network for Precipitation Nowcasting](https://mlsanity.com/paper/2603.04461/) (2026)
- [SmaAT-QMix-UNet: A Parameter-Efficient Vector-Quantized UNet for Precipitation Nowcasting](https://mlsanity.com/paper/2603.21879/) (2026)
- [A Diffusion-Contrastive Graph Neural Network with Virtual Nodes for Wind Nowcasting in Unobserved Regions](https://mlsanity.com/paper/2604.10328/) (2026)
- [VMU-Diff: A Coarse-to-fine Multi-source Data Fusion Framework for Precipitation Nowcasting](https://mlsanity.com/paper/2605.14597/) (2026)
- [MambaRain: Multi-Scale Mamba-Attention Framework for 0-3 Hour Precipitation Nowcasting](https://mlsanity.com/paper/2605.14606/) (2026)
- [Beyond MSE: Improving Precipitation Nowcasting with Multi-Quantile Regression](https://mlsanity.com/paper/2605.30122/) (2026)
- [Temporal Context Conditioning for Seasonality-Aware Precipitation Nowcasting of High-Intensity Rainfall](https://mlsanity.com/paper/2606.09959/) (2026)

## Abstract (arXiv)

Data driven modeling based approaches have recently gained a lot of attention in many challenging meteorological applications including weather element forecasting. This paper introduces a novel data-driven predictive model based on TransUNet for precipitation nowcasting task. The TransUNet model which combines the Transformer and U-Net models has been previously successfully applied in medical segmentation tasks. Here, TransUNet is used as a core model and is further equipped with Convolutional Block Attention Modules (CBAM) and Depthwise-separable Convolution (DSC). The proposed Attention Augmented TransUNet (AA-TransUNet) model is evaluated on two distinct datasets: the Dutch precipitation map dataset and the French cloud cover dataset. The obtained results show that the proposed model outperforms other examined models on both tested datasets. Furthermore, the uncertainty analysis of the proposed AA-TransUNet is provided to give additional insights on its predictions.