# Spatial-Temporal Cross-View Contrastive Pre-training for Check-in Sequence Representation Learning

arXiv: https://arxiv.org/abs/2407.15899 · HTML: https://mlsanity.com/paper/2407.15899/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 41 references that are in the map.
- First posted on arXiv 22 Jul 2024.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [Transformer Hawkes Process](https://mlsanity.com/paper/2002.09291/) (2020)
- [ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer](https://mlsanity.com/paper/2105.11741/) (2021)
- [Intensity-Free Learning of Temporal Point Processes](https://mlsanity.com/paper/1909.12127/) (2019)
- [STAN: Spatio-Temporal Attention Network for Next Location Recommendation](https://mlsanity.com/paper/2102.04095/) (2021)
- [ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embedding](https://mlsanity.com/paper/2109.04380/) (2021)
- [Multi-level Feature Learning for Contrastive Multi-view Clustering](https://mlsanity.com/paper/2106.11193/) (2021)
- [GETNext: Trajectory Flow Map Enhanced Transformer for Next POI Recommendation](https://mlsanity.com/paper/2303.04741/) (2023)
- [Contrastive Learning with Adversarial Perturbations for Conditional Text Generation](https://mlsanity.com/paper/2012.07280/) (2020)
- [Spatio-temporal Diffusion Point Processes](https://mlsanity.com/paper/2305.12403/) (2023)
- [Region Embedding with Intra and Inter-View Contrastive Learning](https://mlsanity.com/paper/2211.08975/) (2022)
- [Virtual Augmentation Supported Contrastive Learning of Sentence Representations](https://mlsanity.com/paper/2110.08552/) (2021)
- [Learning Temporal Point Processes for Efficient Retrieval of Continuous Time Event Sequences](https://mlsanity.com/paper/2202.11485/) (2022)
- [LightMove: A Lightweight Next-POI Recommendation for Taxicab Rooftop Advertising](https://mlsanity.com/paper/2108.04993/) (2021)

## Built on by (most cited first, 2 of 2)

- [Mobility-LLM: Learning Visiting Intentions and Travel Preferences from Human Mobility Data with Large Language Models](https://mlsanity.com/paper/2411.00823/) (2024)
- [When Correlations Mislead: Confounder-Aware Multi-View Urban Region Representation Learning](https://mlsanity.com/paper/2609.15305/) (2026)

## Abstract (arXiv)

The rapid growth of location-based services (LBS) has yielded massive amounts of data on human mobility. Effectively extracting meaningful representations for user-generated check-in sequences is pivotal for facilitating various downstream services. However, the user-generated check-in data are simultaneously influenced by the surrounding objective circumstances and the user's subjective intention. Specifically, the temporal uncertainty and spatial diversity exhibited in check-in data make it difficult to capture the macroscopic spatial-temporal patterns of users and to understand the semantics of user mobility activities. Furthermore, the distinct characteristics of the temporal and spatial information in check-in sequences call for an effective fusion method to incorporate these two types of information. In this paper, we propose a novel Spatial-Temporal Cross-view Contrastive Representation (STCCR) framework for check-in sequence representation learning. Specifically, STCCR addresses the above challenges by employing self-supervision from "spatial topic" and "temporal intention" views, facilitating effective fusion of spatial and temporal information at the semantic level. Besides, STCCR leverages contrastive clustering to uncover users' shared spatial topics from diverse mobility activities, while employing angular momentum contrast to mitigate the impact of temporal uncertainty and noise. We extensively evaluate STCCR on three real-world datasets and demonstrate its superior performance across three downstream tasks.