mlsanityData to 14 Oct 2026

MotifGen: Spatiotemporal interpolation of misaligned satellite images via multi-source generative modeling, in an application to tropical cyclones

arXiv 2606.24263 ↗First posted 23 Jun 2026cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Microwave satellite imagery plays a crucial role in monitoring tropical cyclone precipitation and intensity worldwide, but suffers from long revisit times, potentially missing rapid storm evolution phases. While this raises the need for an interpolation method, it is made challenging by the high level of heterogeneity of microwave data coming from different instruments. In this work, we introduce the first generative model that can be applied to multiple geospatial sources that change across samples, occur at irregular time intervals, are misaligned geographically, and come from instruments with varying characteristics. We apply this model to the case of spatio-temporal interpolation of tropical cyclone microwave images from other microwave and infrared instruments. We train using a self-supervised task in which a random source is masked and reconstructed, and show that it leads to a significant decrease in Continuous Ranked Probability Score over supervised training. We show a further improvement by combining infrared and microwave data compared to microwave only. Using these improvements, the generative model produces an ensemble mean on par with that of a deterministic model, while generating a power spectrum significantly closer to that of true observations. To the best of our knowledge, this is the first generative model that interpolates microwave images of cyclones by combining multiple microwave instruments and infrared observations at irregular time intervals.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 36 references that are in the map. First posted on arXiv 23 Jun 2026.

Builds on

19 of 36 references.

References · 19

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2022Scalable Diffusion Models with Transformerscited by 6,054
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  7. 2018Self-Attention with Relative Position Representationscited by 1,120
  8. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 408
  9. 2023SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagerycited by 151
  10. 2023Residual Corrective Diffusion Modeling for Km-scale Atmospheric Downscalingcited by 110
  11. 2025TerraMind: Large-Scale Generative Multimodality for Earth Observationcited by 104
  12. 2024Mission Critical -- Satellite Data is a Distinct Modality in Machine Learningcited by 77
7 more
  1. 2024OmniSat: Self-Supervised Modality Fusion for Earth Observationcited by 63
  2. 2024ArchesWeather & ArchesWeatherGen: a deterministic and generative model for efficient ML weather forecastingcited by 32
  3. 2023Precipitation nowcasting with generative diffusion modelscited by 23
  4. 2024Wind speed super-resolution and validation: from ERA5 to CERRA via diffusion modelscited by 15
  5. 2024Precipitation Nowcasting Using Diffusion Transformer with Causal Attentioncited by 6
  6. 2025MODS: Multi-source Observations Conditional Diffusion Model for Meteorological State Downscalingcited by 3
  7. 2023Evaluation of drain, a deep-learning approach to rain retrieval from gpm passive microwave radiometercited by 2

Built on by

Not cited yet.