mlsanityData to 14 Oct 2026

SSIN: Self-Supervised Learning for Rainfall Spatial Interpolation

arXiv 2311.15530 ↗First posted 27 Nov 2023cs.LG cs.AI physics.ao-phcited by 4

Not on the map. Why?

Abstract (arXiv)

The acquisition of accurate rainfall distribution in space is an important task in hydrological analysis and natural disaster pre-warning. However, it is impossible to install rain gauges on every corner. Spatial interpolation is a common way to infer rainfall distribution based on available raingauge data. However, the existing works rely on some unrealistic pre-settings to capture spatial correlations, which limits their performance in real scenarios. To tackle this issue, we propose the SSIN, which is a novel data-driven self-supervised learning framework for rainfall spatial interpolation by mining latent spatial patterns from historical observation data. Inspired by the Cloze task and BERT, we fully consider the characteristics of spatial interpolation and design the SpaFormer model based on the Transformer architecture as the core of SSIN. Our main idea is: by constructing rich self-supervision signals via random masking, SpaFormer can learn informative embeddings for raw data and then adaptively model spatial correlations based on rainfall spatial context. Extensive experiments on two real-world raingauge datasets show that our method outperforms the state-of-the-art solutions. In addition, we take traffic spatial interpolation as another use case to further explore the performance of our method, and SpaFormer achieves the best performance on one large real-world traffic dataset, which further confirms the effectiveness and generality of our method.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 15 of its 34 references that are in the map. First posted on arXiv 27 Nov 2023.

Builds on

15 of 34 references.

References · 15

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  6. 2018Self-Attention with Relative Position Representationscited by 1,120
  7. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  8. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 676
  9. 2018AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networkscited by 320
  10. 2020S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximizationcited by 311
  11. 2020Rethinking Positional Encoding in Language Pre-trainingcited by 197
  12. 2021Efficient Self-supervised Vision Transformers for Representation Learningcited by 149
3 more
  1. 2020Inductive Graph Neural Networks for Spatiotemporal Krigingcited by 91
  2. 2023Kriging Convolutional Networkscited by 31
  3. 2021DB-BERT: a Database Tuning Tool that "Reads the Manual"cited by 19

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.