# VITA: Variational Pretraining of Transformers for Climate-Robust Crop Yield Forecasting

arXiv: https://arxiv.org/abs/2508.03589 · HTML: https://mlsanity.com/paper/2508.03589/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 52 references that are in the map.
- First posted on arXiv 5 Aug 2025.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [A CNN-RNN Framework for Crop Yield Prediction](https://mlsanity.com/paper/1911.09045/) (2019)
- [A GNN-RNN Approach for Harnessing Geospatial and Temporal Information: Application to Crop Yield Prediction](https://mlsanity.com/paper/2111.08900/) (2021)
- [An Open and Large-Scale Dataset for Multi-Modal Climate Change-aware Crop Yield Predictions](https://mlsanity.com/paper/2406.06081/) (2024)
- [A Scalable Machine Learning System for Pre-Season Agriculture Yield Forecast](https://mlsanity.com/paper/1806.09244/) (2018)
- [A Variational U-Net for Weather Forecasting](https://mlsanity.com/paper/2111.03476/) (2021)
- [Corn Yield Prediction based on Remotely Sensed Variables Using Variational Autoencoder and Multiple Instance Regression](https://mlsanity.com/paper/2211.13286/) (2022)
- [WeatherFormer: A Pretrained Encoder Model for Learning Robust Weather Representations from Small Datasets](https://mlsanity.com/paper/2405.17455/) (2024)

## Built on by (most cited first, 1 of 1)

- [Evaluating and improving crop-yield forecasting methods during extreme drought](https://mlsanity.com/paper/2608.17971/) (2026)

## Abstract (arXiv)

Accurate crop yield forecasting is essential for global food security. However, current AI models systematically underperform when yields deviate from historical trends. We attribute this to the lack of rich, physically grounded datasets directly linking atmospheric states to yields. To address this, we introduce VITA (Variational Inference Transformer for Asymmetric Data), a variational pretraining framework that learns representations from large satellite-based weather datasets and transfers to the ground-based limited measurements available for yield prediction. VITA is trained using detailed meteorological variables as proxy targets during pretraining and learns to predict latent atmospheric states under a seasonality-aware sinusoidal prior. This allows the model to be fine-tuned using limited weather statistics during deployment. Applied to 763 counties in the US Corn Belt, VITA achieves state-of-the-art performance in predicting corn and soybean yields across all evaluation scenarios, particularly during extreme years, with statistically significant improvements (paired t-test, p < 0.0001). Importantly, VITA outperforms prior frameworks like GNN-RNN without soil data, and larger foundational models (e.g., Chronos-Bolt) with less compute, making it practical for real-world use, especially in data-scarce regions. This work highlights how domain-aware AI design can overcome data limitations and support resilient agricultural forecasting in a changing climate.