# MotifGen: Spatiotemporal interpolation of misaligned satellite images via multi-source generative modeling, in an application to tropical cyclones

arXiv: https://arxiv.org/abs/2606.24263 · HTML: https://mlsanity.com/paper/2606.24263/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 23 Jun 2026.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery](https://mlsanity.com/paper/2207.08051/) (2022)
- [SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery](https://mlsanity.com/paper/2311.17179/) (2023)
- [Residual Corrective Diffusion Modeling for Km-scale Atmospheric Downscaling](https://mlsanity.com/paper/2309.15214/) (2023)
- [TerraMind: Large-Scale Generative Multimodality for Earth Observation](https://mlsanity.com/paper/2504.11171/) (2025)
- [Mission Critical -- Satellite Data is a Distinct Modality in Machine Learning](https://mlsanity.com/paper/2402.01444/) (2024)
- [OmniSat: Self-Supervised Modality Fusion for Earth Observation](https://mlsanity.com/paper/2404.08351/) (2024)
- [ArchesWeather & ArchesWeatherGen: a deterministic and generative model for efficient ML weather forecasting](https://mlsanity.com/paper/2412.12971/) (2024)
- [Precipitation nowcasting with generative diffusion models](https://mlsanity.com/paper/2308.06733/) (2023)
- [Wind speed super-resolution and validation: from ERA5 to CERRA via diffusion models](https://mlsanity.com/paper/2401.15469/) (2024)
- [Precipitation Nowcasting Using Diffusion Transformer with Causal Attention](https://mlsanity.com/paper/2410.13314/) (2024)
- [MODS: Multi-source Observations Conditional Diffusion Model for Meteorological State Downscaling](https://mlsanity.com/paper/2506.14798/) (2025)
- [Evaluation of drain, a deep-learning approach to rain retrieval from gpm passive microwave radiometer](https://mlsanity.com/paper/2303.01220/) (2023)

## Abstract (arXiv)

Microwave satellite imagery plays a crucial role in monitoring tropical cyclone precipitation and intensity worldwide, but suffers from long revisit times, potentially missing rapid storm evolution phases. While this raises the need for an interpolation method, it is made challenging by the high level of heterogeneity of microwave data coming from different instruments. In this work, we introduce the first generative model that can be applied to multiple geospatial sources that change across samples, occur at irregular time intervals, are misaligned geographically, and come from instruments with varying characteristics. We apply this model to the case of spatio-temporal interpolation of tropical cyclone microwave images from other microwave and infrared instruments. We train using a self-supervised task in which a random source is masked and reconstructed, and show that it leads to a significant decrease in Continuous Ranked Probability Score over supervised training. We show a further improvement by combining infrared and microwave data compared to microwave only. Using these improvements, the generative model produces an ensemble mean on par with that of a deterministic model, while generating a power spectrum significantly closer to that of true observations. To the best of our knowledge, this is the first generative model that interpolates microwave images of cyclones by combining multiple microwave instruments and infrared observations at irregular time intervals.