# Self-Supervised Representation Learning for Geospatial Objects: A Survey arXiv: https://arxiv.org/abs/2408.12133 ยท HTML: https://mlsanity.com/paper/2408.12133/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Builds on 116 of its 226 references that are in the map. - First posted on arXiv 22 Aug 2024. ## Builds on (most cited first, 30 of 116) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022) - [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019) - [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023) - [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021) - [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020) - [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018) - [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019) - [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021) - [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022) - [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014) - [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017) - [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013) - [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024) - [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014) - [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019) - [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019) - [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023) - [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016) - [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015) - [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://mlsanity.com/paper/2205.01917/) (2022) - [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020) - [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018) - [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019) ## Built on by (most cited first, 9 of 9) - [Enhancing Large Language Models for Mobility Analytics with Semantic Location Tokenization](https://mlsanity.com/paper/2506.11109/) (2025) - [A Genealogy of Foundation Models in Remote Sensing](https://mlsanity.com/paper/2504.17177/) (2025) - [MoRA: Mobility as the Backbone for Geospatial Representation Learning at Scale](https://mlsanity.com/paper/2506.01297/) (2025) - [Context-Enhanced Multi-View Trajectory Representation Learning: Bridging the Gap through Self-Supervised Models](https://mlsanity.com/paper/2410.13196/) (2024) - [PCaM: A Progressive Focus Attention-Based Information Fusion Method for Improving Vision Transformer Domain Adaptation](https://mlsanity.com/paper/2506.17232/) (2025) - [NARA: Anchor-Conditioned Representation Learning for Heterogeneous Vector Geoentities](https://mlsanity.com/paper/2605.12276/) (2026) - [MoRAX: Mobility-based Representation Augmentation for Geospatial Foundation Models](https://mlsanity.com/paper/2608.17848/) (2026) - [Industrial Kinematic Trajectory Model (IKTM): Coordinate-Free Autoregressive Generator](https://mlsanity.com/paper/2609.22173/) (2026) - [PlacePulse: Learning Transferable Place Representations from Structural Geographic Data](https://mlsanity.com/paper/2610.13390/) (2026) ## Abstract (arXiv) The proliferation of various data sources in urban and territorial environments has significantly facilitated the development of geospatial artificial intelligence (GeoAI) across a wide range of geospatial applications. However, geospatial data, which is inherently linked to geospatial objects, often exhibits data heterogeneity that necessitates specialized fusion and representation strategies while simultaneously being inherently sparse in labels for downstream tasks. Consequently, there is a growing demand for techniques that can effectively leverage geospatial data without heavy reliance on task-specific labels and model designs. This need aligns with the principles of self-supervised learning (SSL), which has garnered increasing attention for its ability to learn effective and generalizable representations directly from data without extensive labeled supervision. This paper presents a comprehensive and up-to-date survey of SSL techniques specifically applied to or developed for geospatial objects in three primary vector geometric types: Point, Polyline, and Polygon. We systematically categorize various SSL techniques into predictive and contrastive methods, and analyze their adaptation to different data types for representation learning across various downstream tasks. Furthermore, we examine the emerging trends in SSL for geospatial objects, particularly the gradual advancements towards geospatial foundation models. Finally, we discuss key challenges in current research and outline promising directions for future investigation. By offering a structured analysis of existing studies, this paper aims to inspire continued progress in integrating SSL with geospatial objects, and the development of geospatial foundation models in a longer term.