# Transforming the Use of Earth Observation Data: Exascale Training of a Generative Compression Model with Historical Priors for up to 10,000x Data Reduction

arXiv: https://arxiv.org/abs/2605.08633 · HTML: https://mlsanity.com/paper/2605.08633/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 49 references that are in the map.
- First posted on arXiv 9 May 2026.

## Builds on (most cited first, 18 of 18)

- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM](https://mlsanity.com/paper/2104.04473/) (2021)
- [Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model](https://mlsanity.com/paper/2201.11990/) (2022)
- [High-Fidelity Generative Image Compression](https://mlsanity.com/paper/2006.09965/) (2020)
- [SpectralGPT: Spectral Remote Sensing Foundation Model](https://mlsanity.com/paper/2311.07113/) (2023)
- [Learned Image Compression with Mixed Transformer-CNN Architectures](https://mlsanity.com/paper/2303.14978/) (2023)
- [MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs](https://mlsanity.com/paper/2402.15627/) (2024)
- [EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling](https://mlsanity.com/paper/2502.09509/) (2025)
- [MetaEarth: A Generative Foundation Model for Global-Scale Remote Sensing Image Generation](https://mlsanity.com/paper/2405.13570/) (2024)
- [SkySense V2: A Unified Foundation Model for Multi-modal Remote Sensing](https://mlsanity.com/paper/2507.13812/) (2025)
- [AxoNN: An asynchronous, message-driven parallel framework for extreme-scale deep learning](https://mlsanity.com/paper/2110.13005/) (2021)
- [Optimizing Distributed Training on Frontier for Large Language Models](https://mlsanity.com/paper/2312.12705/) (2023)
- [ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling](https://mlsanity.com/paper/2505.04802/) (2025)
- [TinyRS-R1: Compact Multimodal Language Model for Remote Sensing](https://mlsanity.com/paper/2505.12099/) (2025)
- [Earth+: on-board satellite imagery compression leveraging historical earth observations](https://mlsanity.com/paper/2403.11434/) (2024)
- [Map-Assisted Remote-Sensing Image Compression at Extremely Low Bitrates](https://mlsanity.com/paper/2409.01935/) (2024)
- [Adaptive and Scalable Compression of Multispectral Images using VVC](https://mlsanity.com/paper/2301.04117/) (2023)

## Abstract (arXiv)

Earth observation is becoming one of the largest data-producing activities in science, yet current pipelines still treat compression as a storage and transmission tool rather than a new way to use data. We present a generative compression framework that learns from historical Earth observation archives and enables on-demand 100x to 10,000x data reduction across downstream tasks. Unlike general visual data, Earth observation repeatedly measures the same evolving planet, making historical-prior learning feasible for extreme compression. To realize this paradigm, we train large generative compression models at exascale on the LineShine Armv9 CPU supercomputer, with co-optimization across model design, kernels, memory hierarchy, runtime, and parallelism. Our implementation sustains 1.54 EFLOP/s and peaks at 2.16 EFLOP/s in end-to-end training. This work shows that historical-prior generative compression can turn Earth observation data into an active, task-adaptive foundation for acquisition, delivery, storage, and scientific use.