# Vision Transformers for Multi-Variable Climate Downscaling: Emulating Regional Climate Models with a Shared Encoder and Multi-Decoder Architecture

arXiv: https://arxiv.org/abs/2506.22447 · HTML: https://mlsanity.com/paper/2506.22447/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 60 references that are in the map.
- First posted on arXiv 12 Jun 2025.

## Builds on (most cited first, 28 of 28)

- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Task Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2009.09796/) (2020)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Are GANs Created Equal? A Large-Scale Study](https://mlsanity.com/paper/1711.10337/) (2017)
- [Transformer for Single Image Super-Resolution](https://mlsanity.com/paper/2108.11084/) (2021)
- [Video Super-Resolution Transformer](https://mlsanity.com/paper/2106.06847/) (2021)
- [Troubling Trends in Machine Learning Scholarship](https://mlsanity.com/paper/1807.03341/) (2018)
- [DeepSD: Generating High Resolution Climate Change Projections through Single Image Super-Resolution](https://mlsanity.com/paper/1703.03126/) (2017)
- [A Generative Deep Learning Approach to Stochastic Downscaling of Precipitation Forecasts](https://mlsanity.com/paper/2204.02028/) (2022)
- [Hard-Constrained Deep Learning for Climate Downscaling](https://mlsanity.com/paper/2208.05424/) (2022)
- [Fourier Neural Operators for Arbitrary Resolution Climate Data Downscaling](https://mlsanity.com/paper/2305.14452/) (2023)
- [Generative Diffusion-based Downscaling for Climate](https://mlsanity.com/paper/2404.17752/) (2024)
- [Precipitation nowcasting with generative diffusion models](https://mlsanity.com/paper/2308.06733/) (2023)
- [Conditional diffusion models for downscaling and bias correction of Earth system model precipitation](https://mlsanity.com/paper/2404.14416/) (2024)
- [ITTR: Unpaired Image-to-Image Translation with Transformers](https://mlsanity.com/paper/2203.16015/) (2022)
- [Wind speed super-resolution and validation: from ERA5 to CERRA via diffusion models](https://mlsanity.com/paper/2401.15469/) (2024)
- [RainNet: A Large-Scale Imagery Dataset and Benchmark for Spatial Precipitation Downscaling](https://mlsanity.com/paper/2012.09700/) (2020)
- [Can AI be enabled to dynamical downscaling? A Latent Diffusion Model to mimic km-scale COSMO5.0CLM9 simulations](https://mlsanity.com/paper/2406.13627/) (2024)
- [On the modern deep learning approaches for precipitation downscaling](https://mlsanity.com/paper/2207.00808/) (2022)
- [Deep generative model super-resolves spatially correlated multiregional climate data](https://mlsanity.com/paper/2209.12433/) (2022)
- [Regional climate projections using a deep-learning-based model-ranking and downscaling framework: Application to European climate zones](https://mlsanity.com/paper/2502.20132/) (2025)
- [DSAF: A Dual-Stage Adaptive Framework for Numerical Weather Prediction Downscaling](https://mlsanity.com/paper/2312.12476/) (2023)
- [Controlling Ensemble Variance in Diffusion Models: An Application for Reanalyses Downscaling](https://mlsanity.com/paper/2501.14822/) (2025)

## Abstract (arXiv)

Global Climate Models (GCMs) are critical for simulating large-scale climate dynamics, but their coarse spatial resolution limits their applicability in regional studies. Regional Climate Models (RCMs) address this limitation through dynamical downscaling, albeit at considerable computational cost and with limited flexibility. Deep learning has emerged as an efficient data-driven alternative; however, most existing approaches focus on single-variable models that downscale one variable at a time. This paradigm can lead to redundant computation, limited contextual awareness, and weak cross-variable interactions.To address these limitations, we propose a multi-variable Vision Transformer (ViT) architecture with a shared encoder and variable-specific decoders (1EMD). The proposed model jointly predicts six key climate variables: surface temperature, wind speed, 500 hPa geopotential height, total precipitation, surface downwelling shortwave radiation, and surface downwelling longwave radiation, directly from GCM-resolution inputs, emulating RCM-scale downscaling over Europe. Compared to single-variable ViT models, the 1EMD architecture improves performance across all six variables, achieving an average MSE reduction of approximately 5.5% under a fair and controlled comparison. It also consistently outperforms alternative multi-variable baselines, including a single-decoder ViT and a multi-variable U-Net. Moreover, multi-variable models substantially reduce computational cost, yielding a 29-32% lower inference time per variable compared to single-variable approaches. Overall, our results demonstrate that multi-variable modeling provides systematic advantages for high-resolution climate downscaling in terms of both accuracy and efficiency. Among the evaluated architectures, the proposed 1EMD ViT achieves the most favorable trade-off between predictive performance and computational cost.