# Temporal Embeddings: Scalable Self-Supervised Temporal Representation Learning from Spatiotemporal Data for Multimodal Computer Vision

arXiv: https://arxiv.org/abs/2401.08581 · HTML: https://mlsanity.com/paper/2401.08581/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 8 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 5 of 5)

- [VAE-Info-cGAN: Generating Synthetic Images by Combining Pixel-level and Feature-level Geospatial Conditional Inputs](https://mlsanity.com/paper/2012.04196/) (2020)
- [Trinity: A No-Code AI platform for complex spatial datasets](https://mlsanity.com/paper/2106.11756/) (2021)
- [SeMAnD: Self-Supervised Anomaly Detection in Multimodal Geospatial Datasets](https://mlsanity.com/paper/2309.15245/) (2023)
- [Reachability Embeddings: Scalable Self-Supervised Representation Learning from Mobility Trajectories for Multimodal Geospatial Computer Vision](https://mlsanity.com/paper/2110.12521/) (2021)
- [Self-Supervised Temporal Analysis of Spatiotemporal Data](https://mlsanity.com/paper/2304.13143/) (2023)

## Abstract (arXiv)

There exists a correlation between geospatial activity temporal patterns and type of land use. A novel self-supervised approach is proposed to stratify landscape based on mobility activity time series. First, the time series signal is transformed to the frequency domain and then compressed into task-agnostic temporal embeddings by a contractive autoencoder, which preserves cyclic temporal patterns observed in time series. The pixel-wise embeddings are converted to image-like channels that can be used for task-based, multimodal modeling of downstream geospatial tasks using deep semantic segmentation. Experiments show that temporal embeddings are semantically meaningful representations of time series data and are effective across different tasks such as classifying residential area and commercial areas. Temporal embeddings transform sequential, spatiotemporal motion trajectory data into semantically meaningful image-like tensor representations that can be combined (multimodal fusion) with other data modalities that are or can be transformed into image-like tensor representations (for e.g., RBG imagery, graph embeddings of road networks, passively collected imagery like SAR, etc.) to facilitate multimodal learning in geospatial computer vision. Multimodal computer vision is critical for training machine learning models for geospatial feature detection to keep a geospatial mapping service up-to-date in real-time and can significantly improve user experience and above all, user safety.