# Global atmospheric data assimilation with multi-modal masked autoencoders

arXiv: https://arxiv.org/abs/2407.11696 · HTML: https://mlsanity.com/paper/2407.11696/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 34 references that are in the map.
- First posted on arXiv 16 Jul 2024.

## Builds on (most cited first, 9 of 9)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Masked Autoencoders As Spatiotemporal Learners](https://mlsanity.com/paper/2205.09113/) (2022)
- [Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning](https://mlsanity.com/paper/2212.14532/) (2022)
- [MADE: Masked Autoencoder for Distribution Estimation](https://mlsanity.com/paper/1502.03509/) (2015)
- [MultiMAE: Multi-modal Multi-task Masked Autoencoders](https://mlsanity.com/paper/2204.01678/) (2022)
- [4M: Massively Multimodal Masked Modeling](https://mlsanity.com/paper/2312.06647/) (2023)
- [Combining data assimilation and machine learning to infer unresolved scale parametrisation](https://mlsanity.com/paper/2009.04318/) (2020)
- [On Masked Pre-training and the Marginal Likelihood](https://mlsanity.com/paper/2306.00520/) (2023)

## Built on by (most cited first, 11 of 11)

- [Prithvi WxC: Foundation Model for Weather and Climate](https://mlsanity.com/paper/2409.13598/) (2024)
- [Deep Learning and Foundation Models for Weather Prediction: A Survey](https://mlsanity.com/paper/2501.06907/) (2025)
- [Satellite Observations Guided Diffusion Model for Accurate Meteorological States at Arbitrary Resolution](https://mlsanity.com/paper/2502.07814/) (2025)
- [MODS: Multi-source Observations Conditional Diffusion Model for Meteorological State Downscaling](https://mlsanity.com/paper/2506.14798/) (2025)
- [Learning more physically realistic dynamics in machine-learning based weather forecasting with latent-space constraints](https://mlsanity.com/paper/2510.04006/) (2025)
- [DAWP: A framework for global observation forecasting via Data Assimilation and Weather Prediction in satellite observation space](https://mlsanity.com/paper/2510.15978/) (2025)
- [Contrastive Learning Boosts Deterministic and Generative Models for Weather Data](https://mlsanity.com/paper/2603.24744/) (2026)
- [Earth-o1: A Grid-free Observation-native Atmospheric World Model](https://mlsanity.com/paper/2605.06337/) (2026)
- [New Tools are Needed for Tracking Adherence to AI Model Behavioral Use Clauses](https://mlsanity.com/paper/2505.22287/) (2025)
- [FlowDA: Accurate, Low-Latency Weather Data Assimilation via Flow Matching](https://mlsanity.com/paper/2602.06800/) (2026)
- [C3DIR: A Deep Learning 3-Dimensional Cloud Property Retrieval Scheme for Passive Satellite Imagers](https://mlsanity.com/paper/2607.16929/) (2026)

## Abstract (arXiv)

Global data assimilation enables weather forecasting at all scales and provides valuable data for studying the Earth system. However, the computational demands of physics-based algorithms used in operational systems limits the volume and diversity of observations that are assimilated. Here, we present "EarthNet", a multi-modal foundation model for data assimilation that learns to predict a global gap-filled atmospheric state solely from satellite observations. EarthNet is trained as a masked autoencoder that ingests a 12 hour sequence of observations and learns to fill missing data from other sensors. We show that EarthNet performs a form of data assimilation producing a global 0.16 degree reanalysis dataset of 3D atmospheric temperature and humidity at a fraction of the time compared to operational systems. It is shown that the resulting reanalysis dataset reproduces climatology by evaluating a 1 hour forecast background state against observations. We also show that our 3D humidity predictions outperform MERRA-2 and ERA5 reanalyses by 10% to 60% between the middle troposphere and lower stratosphere (5 to 20 km altitude) and our 3D temperature and humidity are statistically equivalent to the Microwave integrated Retrieval System (MiRS) observations at nearly every level of the atmosphere. Our results indicate significant promise in using EarthNet for high-frequency data assimilation and global weather forecasting.