# CDSA: Cross-Dimensional Self-Attention for Multivariate, Geo-tagged Time Series Imputation

arXiv: https://arxiv.org/abs/1905.09904 · HTML: https://mlsanity.com/paper/1905.09904/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2019.

## Built on by (most cited first, 24 of 24)

- [SAITS: Self-Attention-based Imputation for Time Series](https://mlsanity.com/paper/2202.08516/) (2022)
- [PriSTI: A Conditional Diffusion Framework for Spatiotemporal Imputation](https://mlsanity.com/paper/2302.09746/) (2023)
- [Ti-MAE: Self-Supervised Masked Time Series Autoencoders](https://mlsanity.com/paper/2301.08871/) (2023)
- [Deep Learning for Multivariate Time Series Imputation: A Survey](https://mlsanity.com/paper/2402.04059/) (2024)
- [MTS-Mixers: Multivariate Time Series Forecasting via Factorized Temporal and Channel Mixing](https://mlsanity.com/paper/2302.04501/) (2023)
- [ImputeFormer: Low Rankness-Induced Transformers for Generalizable Spatiotemporal Imputation](https://mlsanity.com/paper/2312.01728/) (2023)
- [TSI-Bench: Benchmarking Time Series Imputation](https://mlsanity.com/paper/2406.12747/) (2024)
- [NRTSI: Non-Recurrent Time Series Imputation](https://mlsanity.com/paper/2102.03340/) (2021)
- [TempCLR: Temporal Alignment Representation with Contrastive Learning](https://mlsanity.com/paper/2212.13738/) (2022)
- [Graph-based Virtual Sensing from Sparse and Partial Multivariate Observations](https://mlsanity.com/paper/2402.12598/) (2024)
- [Pedestrian Trajectory Prediction with Missing Data: Datasets, Imputation, and Benchmarking](https://mlsanity.com/paper/2411.00174/) (2024)
- [Continuous-time Autoencoders for Regular and Irregular Time Series Imputation](https://mlsanity.com/paper/2312.16581/) (2023)
- [STAMImputer: Spatio-Temporal Attention MoE for Traffic Data Imputation](https://mlsanity.com/paper/2506.08054/) (2025)
- [Temporal Saliency Detection Towards Explainable Transformer-based Timeseries Forecasting](https://mlsanity.com/paper/2212.07771/) (2022)
- [Filling out the missing gaps: Time Series Imputation with Semi-Supervised Learning](https://mlsanity.com/paper/2304.04275/) (2023)
- [RITA: Group Attention is All You Need for Timeseries Analytics](https://mlsanity.com/paper/2306.01926/) (2023)
- [EdgeConvFormer: Dynamic Graph CNN and Transformer based Anomaly Detection in Multivariate Time Series](https://mlsanity.com/paper/2312.01729/) (2023)
- [Scalable Numerical Embeddings for Multivariate Time Series: Enhancing Healthcare Data Representation Learning](https://mlsanity.com/paper/2405.16557/) (2024)
- [Model-Attentive Ensemble Learning for Sequence Modeling](https://mlsanity.com/paper/2102.11500/) (2021)
- [Multistage Large Segment Imputation Framework Based on Deep Learning and Statistic Metrics](https://mlsanity.com/paper/2209.11766/) (2022)
- [ProDiff: Prototype-Guided Diffusion for Minimal Information Trajectory Imputation](https://mlsanity.com/paper/2505.23048/) (2025)
- [Impute With Confidence: A Framework for Uncertainty Aware Multivariate Time Series Imputation](https://mlsanity.com/paper/2507.09353/) (2025)
- [STDiff: A State Transition Diffusion Framework for Time Series Imputation in Industrial Systems](https://mlsanity.com/paper/2508.19011/) (2025)
- [Element2Vec: Build Chemical Element Representation from Text for Property Prediction](https://mlsanity.com/paper/2510.13916/) (2025)

## Abstract (arXiv)

Many real-world applications involve multivariate, geo-tagged time series data: at each location, multiple sensors record corresponding measurements. For example, air quality monitoring system records PM2.5, CO, etc. The resulting time-series data often has missing values due to device outages or communication errors. In order to impute the missing values, state-of-the-art methods are built on Recurrent Neural Networks (RNN), which process each time stamp sequentially, prohibiting the direct modeling of the relationship between distant time stamps. Recently, the self-attention mechanism has been proposed for sequence modeling tasks such as machine translation, significantly outperforming RNN because the relationship between each two time stamps can be modeled explicitly. In this paper, we are the first to adapt the self-attention mechanism for multivariate, geo-tagged time series data. In order to jointly capture the self-attention across multiple dimensions, including time, location and the sensor measurements, while maintain low computational complexity, we propose a novel approach called Cross-Dimensional Self-Attention (CDSA) to process each dimension sequentially, yet in an order-independent manner. Our extensive experiments on four real-world datasets, including three standard benchmarks and our newly collected NYC-traffic dataset, demonstrate that our approach outperforms the state-of-the-art imputation and forecasting methods. A detailed systematic analysis confirms the effectiveness of our design choices.