Compressing high-resolution data through latent representation encoding for downscaling large-scale AI weather forecast model
Not on the map. Why?
Abstract (arXiv)
The rapid advancement of artificial intelligence (AI) in weather research has been driven by the ability to learn from large, high-dimensional datasets. However, this progress also poses significant challenges, particularly regarding the substantial costs associated with processing extensive data and the limitations of computational resources. Inspired by the Neural Image Compression (NIC) task in computer vision, this study seeks to compress weather data to address these challenges and enhance the efficiency of downstream applications. Specifically, we propose a variational autoencoder (VAE) framework tailored for compressing high-resolution datasets, specifically the High Resolution China Meteorological Administration Land Data Assimilation System (HRCLDAS) with a spatial resolution of 1 km. Our framework successfully reduced the storage size of 3 years of HRCLDAS data from 8.61 TB to just 204 GB, while preserving essential information. In addition, we demonstrated the utility of the compressed data through a downscaling task, where the model trained on the compressed dataset achieved accuracy comparable to that of the model trained on the original data. These results highlight the effectiveness and potential of the compressed data for future weather research.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 12 of its 22 references that are in the map. First posted on arXiv 10 Oct 2024.
Builds on
12 of 22 references.
References · 12
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2018Variational image compression with a scale hyperpriorcited by 910
- 2016End-to-end Optimized Image Compressioncited by 673
- 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
- 2020Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modulescited by 570
- 2022GraphCast: Learning skillful medium-range global weather forecastingcited by 261
- 2023FuXi: A cascade machine learning forecasting system for 15-day global weather forecastcited by 237
- 2023WeatherBench 2: A benchmark for the next generation of data-driven global weather modelscited by 227
- 2017Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networkscited by 138
- 2020Stochastic Super-Resolution for Downscaling Time-Evolving Atmospheric Fields with a Generative Adversarial Networkcited by 68
- 2022A Generative Deep Learning Approach to Stochastic Downscaling of Precipitation Forecastscited by 64
- 2024FuXi-2.0: Advancing machine learning weather forecasting model for practical applicationscited by 11
Built on by
Citing papers per year: 2025: 1.
1 paper.