mlsanityData to 14 Oct 2026

Quantizing Space and Time: Fusing Time Series and Images for Earth Observation

arXiv 2510.23118 ↗First posted 27 Oct 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose a task-agnostic framework for multimodal fusion of time series and single timestamp images, enabling cross-modal generation and robust downstream performance. Our approach explores deterministic and learned strategies for time series quantization and then leverages a masked correlation learning objective, aligning discrete image and time series tokens in a unified representation space. Instantiated in the Earth observation domain, the pretrained model generates consistent global temperature profiles from satellite imagery and is validated through counterfactual experiments. Across downstream tasks, our task-agnostic pretraining outperforms task-specific fusion by 6% in R^2 and 2% in RMSE on average, and exceeds baseline methods by 50% in R^2 and 12% in RMSE. Finally, we analyze gradient sensitivity across modalities, providing insights into model robustness. Code, data, and weights will be released under a permissive license.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 25 references that are in the map. First posted on arXiv 27 Oct 2025.

Builds on

14 of 25 references.

References · 14

  1. 2017Neural Discrete Representation Learningcited by 4,712
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  4. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  5. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  6. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 440
  7. 2025TerraMind: Large-Scale Generative Multimodality for Earth Observationcited by 104
  8. 2022MedFuse: Multi-modal fusion with clinical time-series data and chest X-ray imagescited by 54
  9. 2024TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysiscited by 39
  10. 2019A CNN-RNN Framework for Crop Yield Predictioncited by 36
  11. 2023MMST-ViT: Climate Change-aware Crop Yield Prediction via Multi-Modal Spatial-Temporal Vision Transformercited by 29
  12. 2025TerraMesh: A Planetary Mosaic of Multimodal Earth Observation Datacited by 14
2 more
  1. 2024An Open and Large-Scale Dataset for Multi-Modal Climate Change-aware Crop Yield Predictionscited by 11
  2. 2025MSDformer: Multi-scale Discrete Transformer For Time Series Generationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026TriTS: Time Series Forecasting from a Multimodal Perspective
  2. 2025PIPE: Physics-Informed Position Encoding for Alignment of Satellite Images and Time Series
  3. 2026MEOX: Compact Multimodal Mixture-of-Experts for Earth Observation
  4. 2023Does Vector Quantization Fail in Spatio-Temporal Forecasting? Exploring a Differentiable Sparse Soft-Vector Quantization Approach
  5. 2025TAMMs: Change Understanding and Forecasting in Satellite Image Time Series with Temporal-Aware Multimodal Models
  6. 2026COP-GEN: Latent Diffusion Transformer for Copernicus Earth Observation Data
  7. 2026TimeOmni-VL: Unified Models for Time Series Understanding and Generation
  8. 2025Exploring the design space of diffusion and flow models for data fusion