mlsanityData to 14 Oct 2026

From Surface Forecasting to Observability Forecasting: A Latent World Model for Cloud-Aware EO Monitoring

arXiv 2607.13651 ↗First posted 15 Jul 2026cs.CVcited by 0

On the map: World models → Latent predictive world models

Abstract (arXiv)

The bottleneck of Earth Observation processing chains is not the arrival of new imagery but whether the surface is actually visible when the image arrives. We study this as an observability forecasting problem on EarthNet2021. Given recent multispectral imagery and exogenous weather drivers, the goal is to predict whether the next acquisition will be usable and, if not, when a usable view is likely to return. To do this, we adapt LeWorldModel, a joint-embedding predictive architecture world model, to cloud-aware Earth Observation sequences. The final pipeline converts raw minicubes into episodic HDF5 sequences with five image channels (blue, green, red, near-infrared, cloud mask) and eight meteorological and calendar covariates. The resulting model has 18.0M trainable parameters and is trained from scratch on 23,904 training episodes. The trained leWorldModel is evaluated under a locked protocol: linear probes are fit on train only, calibration choices are set on an internal validation split, and the fitted heads are then frozen for valsplit, IID, OOD, and extreme evaluation. On the full frozen-bundle observability benchmark, LeWorldModel consistently outperforms persistence. For next-step usability, balanced accuracy ranges from 0.769 to 0.887, compared with 0.493 to 0.556 for persistence. For exact first-usable-horizon prediction, accuracy ranges from 0.602 to 0.806, compared with 0.120 to 0.369 for persistence. Against a frozen LightGBM baseline fit on the same training windows, LeWorldModel is better on continuous clear/cloud regression and on exact recovery timing on valsplit, IID, and extreme, while LightGBM is stronger on the simpler binary any-usable-within-six task and is more robust on OOD. In separate sampled diagnostic analyses, LeWM also produces strong ranking-based anomaly signals under synthetic temporal inconsistencies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 18 of its 18 references that are in the map. First posted on arXiv 15 Jul 2026.

Builds on

18 of 18 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026EO-WM: A Physically Informed World Model for Probabilistic Earth Observation Forecasting25 Jun
  2. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels13 Mar
  3. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture19 Jan
  4. 2018Learning Latent Dynamics for Planning from Pixels12 Nov

Other references · 14

  1. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  2. 2018World Modelscited by 1,513
  3. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  4. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 408
  5. 2022Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learningcited by 308
  6. 2022Earthformer: Exploring Space-Time Transformers for Earth System Forecastingcited by 152
  7. 2023Foundation Models for Generalist Geospatial Artificial Intelligencecited by 149
  8. 2021EarthNet2021: A large-scale dataset and challenge for Earth surface forecasting as a guided video prediction taskcited by 34
  9. 2022SEN12MS-CR-TS: A Remote Sensing Data Set for Multi-modal Multi-temporal Cloud Removalcited by 32
  10. 2023UnCRtainTS: Uncertainty Quantification for Cloud Removal in Optical Satellite Time Seriescited by 28
  11. 2020EarthNet2021: A novel large-scale dataset and challenge for forecasting localized climate impactscited by 4
  12. 2022Efficient data-driven gap filling of satellite image time series using deep neural networks with partial convolutionscited by 4
2 more
  1. 2025Lightweight Metadata-Aware Mixture-of-Experts Masked Autoencoder for Earth Observationcited by 3
  2. 2024Seeing Through the Clouds: Cloud Gap Imputation with Prithvi Foundation Modelcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Earth-o1: A Grid-free Observation-native Atmospheric World Model
  2. 2026AirCast-SR: A Foundation Model for Kilometer-Scale Atmospheric Super-Resolution via Latent Consistency Diffusion
  3. 2026The Last Remote Sensing Model Built by Humans? Casting Any Earth Observation Task as Recursive Self-Improvement
  4. 2026EO-Gym: A Multimodal, Interactive Environment for Earth Observation Agents
  5. 2026From Nowcasting to Forecasting: Adapting a Reanalysis-Trained
  6. 2025Regression in EO: Are VLMs Up to the Challenge?
  7. 2026Earth-Agent-Pro: Towards Real-World Full-Chain Earth Observation with Agents
  8. 2026Tracing the Unlabeled Storm: Cross-Variable Transfer in a Lagrangian Atmospheric JEPA Framework