mlsanityData to 14 Oct 2026

Self-Supervised Representation Learning for Geospatial Objects: A Survey

arXiv 2408.12133 ↗First posted 22 Aug 2024cs.AI cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

The proliferation of various data sources in urban and territorial environments has significantly facilitated the development of geospatial artificial intelligence (GeoAI) across a wide range of geospatial applications. However, geospatial data, which is inherently linked to geospatial objects, often exhibits data heterogeneity that necessitates specialized fusion and representation strategies while simultaneously being inherently sparse in labels for downstream tasks. Consequently, there is a growing demand for techniques that can effectively leverage geospatial data without heavy reliance on task-specific labels and model designs. This need aligns with the principles of self-supervised learning (SSL), which has garnered increasing attention for its ability to learn effective and generalizable representations directly from data without extensive labeled supervision. This paper presents a comprehensive and up-to-date survey of SSL techniques specifically applied to or developed for geospatial objects in three primary vector geometric types: Point, Polyline, and Polygon. We systematically categorize various SSL techniques into predictive and contrastive methods, and analyze their adaptation to different data types for representation learning across various downstream tasks. Furthermore, we examine the emerging trends in SSL for geospatial objects, particularly the gradual advancements towards geospatial foundation models. Finally, we discuss key challenges in current research and outline promising directions for future investigation. By offering a structured analysis of existing studies, this paper aims to inspire continued progress in integrating SSL with geospatial objects, and the development of geospatial foundation models in a longer term.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 116 of its 226 references that are in the map. First posted on arXiv 22 Aug 2024.

Builds on

116 of 226 references.

References · 116

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2020Language Models are Few-Shot Learnerscited by 32,419
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  7. 2023GPT-4 Technical Reportcited by 18,396
  8. 2022Training language models to follow instructions with human feedbackcited by 15,971
  9. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  10. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  11. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  12. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
38 more
  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  7. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  9. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  10. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  11. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  12. 2023A Survey of Large Language Modelscited by 2,576
  13. 2016node2vec: Scalable Feature Learning for Networkscited by 1,918
  14. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  15. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  16. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  17. 2018Deep Graph Infomaxcited by 1,106
  18. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  19. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  20. 2020Self-supervised Learning: Generative or Contrastivecited by 694
  21. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 657
  22. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  23. 2024Large Language Models: A Surveycited by 525
  24. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 343
  25. 2023Language Models Represent Space and Timecited by 316
  26. 2021Graph Self-Supervised Learning: A Surveycited by 289
  27. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 252
  28. 2020What Should Not Be Contrastive in Contrastive Learningcited by 226
  29. 2023GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localizationcited by 187
  30. 2019Deep Learning for Spatio-Temporal Data Mining: A Surveycited by 172
  31. 2021Self-supervised Learning on Graphs: Contrastive, Generative,or Predictivecited by 136
  32. 2018Tile2Vec: Unsupervised representation learning for spatially distributed datacited by 110
  33. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 99
  34. 2022Self-Supervised Learning for Recommender Systems: A Surveycited by 95
  35. 2020Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learningcited by 90
  36. 2023On the Opportunities and Challenges of Foundation Models for Geospatial Artificial Intelligencecited by 88
  37. 2024UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Predictioncited by 87
  38. 2022Self-Supervised Learning for Videos: A Surveycited by 85

50 most cited of 116 All 116 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 4.

9 papers.

  1. 2025Enhancing Large Language Models for Mobility Analytics with Semantic Location Tokenizationcited by 8
  2. 2025A Genealogy of Foundation Models in Remote Sensingcited by 4
  3. 2025MoRA: Mobility as the Backbone for Geospatial Representation Learning at Scalecited by 4
  4. 2024Context-Enhanced Multi-View Trajectory Representation Learning: Bridging the Gap through Self-Supervised Modelscited by 2
  5. 2025PCaM: A Progressive Focus Attention-Based Information Fusion Method for Improving Vision Transformer Domain Adaptationcited by 0
  6. 2026NARA: Anchor-Conditioned Representation Learning for Heterogeneous Vector Geoentitiescited by 0
  7. 2026MoRAX: Mobility-based Representation Augmentation for Geospatial Foundation Modelscited by 0
  8. 2026Industrial Kinematic Trajectory Model (IKTM): Coordinate-Free Autoregressive Generatorcited by 0
  9. 2026PlacePulse: Learning Transferable Place Representations from Structural Geographic Datacited by 0