# CRA5: Extreme Compression of ERA5 for Portable Global Climate and Weather Research via an Efficient Variational Transformer

arXiv: https://arxiv.org/abs/2405.03376 · HTML: https://mlsanity.com/paper/2405.03376/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 58 references that are in the map.
- First posted on arXiv 6 May 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding](https://mlsanity.com/paper/2203.10886/) (2022)
- [WeatherBench: A benchmark dataset for data-driven weather forecasting](https://mlsanity.com/paper/2002.00469/) (2020)
- [Channel-wise Autoregressive Entropy Models for Learned Image Compression](https://mlsanity.com/paper/2007.08739/) (2020)
- [Learned Image Compression with Mixed Transformer-CNN Architectures](https://mlsanity.com/paper/2303.14978/) (2023)
- [FourCastNet: Accelerating Global High-Resolution Weather Forecasting using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2208.05419/) (2022)
- [The Devil Is in the Details: Window-based Attention for Image Compression](https://mlsanity.com/paper/2203.08450/) (2022)
- [Context-adaptive Entropy Model for End-to-end Optimized Image Compression](https://mlsanity.com/paper/1809.10452/) (2018)
- [Entroformer: A Transformer-based Entropy Model for Learned Image Compression](https://mlsanity.com/paper/2202.05492/) (2022)
- [Enhanced Invertible Encoding for Learned Image Compression](https://mlsanity.com/paper/2108.03690/) (2021)
- [Practical Full Resolution Learned Lossless Image Compression](https://mlsanity.com/paper/1811.12817/) (2018)
- [Improving Deep Video Compression by Resolution-adaptive Flow Coding](https://mlsanity.com/paper/2009.05982/) (2020)
- [Contextformer: A Transformer with Spatio-Channel Attention for Context Modeling in Learned Image Compression](https://mlsanity.com/paper/2203.02452/) (2022)
- [SwinVRNN: A Data-Driven Ensemble Forecasting Model via Learned Distribution Perturbation](https://mlsanity.com/paper/2205.13158/) (2022)
- [Compressing multidimensional weather and climate data into neural networks](https://mlsanity.com/paper/2210.12538/) (2022)
- [FengWu-GHR: Learning the Kilometer-scale Medium-range Global Weather Forecasting](https://mlsanity.com/paper/2402.00059/) (2024)
- [ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast](https://mlsanity.com/paper/2402.01295/) (2024)
- [Clairvoyant Prefetching for Distributed Machine Learning I/O](https://mlsanity.com/paper/2101.08734/) (2021)
- [Global Tropical Cyclone Intensity Forecasting with Multi-modal Multi-scale Causal Autoregressive Model](https://mlsanity.com/paper/2402.13270/) (2024)

## Built on by (most cited first, 22 of 22)

- [HEMGS: A Hybrid Entropy Model for 3D Gaussian Splatting Data Compression](https://mlsanity.com/paper/2411.18473/) (2024)
- [Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedback](https://mlsanity.com/paper/2501.03916/) (2025)
- [Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling](https://mlsanity.com/paper/2405.13796/) (2024)
- [Deep Learning and Foundation Models for Weather Prediction: A Survey](https://mlsanity.com/paper/2501.06907/) (2025)
- [Group-aware Parameter-efficient Updating for Content-Adaptive Neural Video Compression](https://mlsanity.com/paper/2405.04274/) (2024)
- [Lossy Neural Compression for Geospatial Analytics: A Review](https://mlsanity.com/paper/2503.01505/) (2025)
- [PredBench: Benchmarking Spatio-Temporal Prediction across Diverse Disciplines](https://mlsanity.com/paper/2407.08418/) (2024)
- [LaDCast: A Latent Diffusion Model for Medium-Range Ensemble Weather Forecasting](https://mlsanity.com/paper/2506.09193/) (2025)
- [Benchmarking Physics-Informed Time-Series Models for Operational Global Station Weather Forecasting](https://mlsanity.com/paper/2406.14399/) (2024)
- [An Efficient Adaptive Compression Method for Human Perception and Machine Vision Tasks](https://mlsanity.com/paper/2501.04329/) (2025)
- [Align-DA: Align Score-based Atmospheric Data Assimilation with Multiple Preferences](https://mlsanity.com/paper/2505.22008/) (2025)
- [Towards Point Cloud Compression for Machine Perception: A Simple and Strong Baseline by Learning the Octree Depth Level Predictor](https://mlsanity.com/paper/2406.00791/) (2024)
- [Transforming Weather Data from Pixel to Latent Space](https://mlsanity.com/paper/2503.06623/) (2025)
- [Neural Stereo Video Compression with Hybrid Disparity Compensation](https://mlsanity.com/paper/2504.20383/) (2025)
- [DEF: Diffusion-augmented Ensemble Forecasting](https://mlsanity.com/paper/2506.07324/) (2025)
- [Video Compression for Spatiotemporal Earth System Data](https://mlsanity.com/paper/2506.19656/) (2025)
- [DAWP: A framework for global observation forecasting via Data Assimilation and Weather Prediction in satellite observation space](https://mlsanity.com/paper/2510.15978/) (2025)
- [Contrastive Learning Boosts Deterministic and Generative Models for Weather Data](https://mlsanity.com/paper/2603.24744/) (2026)
- [HiHa: Introducing Hierarchical Harmonic Decomposition to Implicit Neural Compression for Atmospheric Data](https://mlsanity.com/paper/2411.06155/) (2024)
- [LLMComp: A Language Modeling Paradigm for Error-Bounded Scientific Data Compression (Technical Report)](https://mlsanity.com/paper/2510.23632/) (2025)
- [Field-Space Autoencoder for Scalable Climate Emulators](https://mlsanity.com/paper/2601.15102/) (2026)
- [Earth Science Foundation Models: From Perception to Reasoning and Discovery](https://mlsanity.com/paper/2605.12542/) (2026)

## Abstract (arXiv)

The advent of data-driven weather forecasting models, which learn from hundreds of terabytes (TB) of reanalysis data, has significantly advanced forecasting capabilities. However, the substantial costs associated with data storage and transmission present a major challenge for data providers and users, affecting resource-constrained researchers and limiting their accessibility to participate in AI-based meteorological research. To mitigate this issue, we introduce an efficient neural codec, the Variational Autoencoder Transformer (VAEformer), for extreme compression of climate data to significantly reduce data storage cost, making AI-based meteorological research portable to researchers. Our approach diverges from recent complex neural codecs by utilizing a low-complexity Auto-Encoder transformer. This encoder produces a quantized latent representation through variance inference, which reparameterizes the latent space as a Gaussian distribution. This method improves the estimation of distributions for cross-entropy coding. Extensive experiments demonstrate that our VAEformer outperforms existing state-of-the-art compression methods in the context of climate data. By applying our VAEformer, we compressed the most popular ERA5 climate dataset (226 TB) into a new dataset, CRA5 (0.7 TB). This translates to a compression ratio of over 300 while retaining the dataset's utility for accurate scientific analysis. Further, downstream experiments show that global weather forecasting models trained on the compact CRA5 dataset achieve forecasting accuracy comparable to the model trained on the original dataset. Code, the CRA5 dataset, and the pre-trained model are available at https://github.com/taohan10200/CRA5.