# SAITS: Self-Attention-based Imputation for Time Series

arXiv: https://arxiv.org/abs/2202.08516 · HTML: https://mlsanity.com/paper/2202.08516/

## Key facts

- Cited by 169 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 62 references that are in the map.
- First posted on arXiv 17 Feb 2022.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [GAIN: Missing Data Imputation using Generative Adversarial Nets](https://mlsanity.com/paper/1806.02920/) (2018)
- [BRITS: Bidirectional Recurrent Imputation for Time Series](https://mlsanity.com/paper/1805.10572/) (2018)
- [GP-VAE: Deep Probabilistic Time Series Imputation](https://mlsanity.com/paper/1907.04155/) (2019)
- [Handling Incomplete Heterogeneous Data using VAEs](https://mlsanity.com/paper/1807.03653/) (2018)
- [Set Functions for Time Series](https://mlsanity.com/paper/1909.12064/) (2019)
- [Estimating Missing Data in Temporal Data Streams Using Multi-directional Recurrent Neural Networks](https://mlsanity.com/paper/1711.08742/) (2017)
- [MisGAN: Learning from Incomplete Data with Generative Adversarial Networks](https://mlsanity.com/paper/1902.09599/) (2019)
- [MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcare](https://mlsanity.com/paper/1810.09593/) (2018)
- [Gaussian Process Prior Variational Autoencoders](https://mlsanity.com/paper/1810.11738/) (2018)
- [DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding](https://mlsanity.com/paper/1709.04696/) (2017)
- [NAOMI: Non-Autoregressive Multiresolution Sequence Imputation](https://mlsanity.com/paper/1901.10946/) (2019)
- [Modeling Continuous Stochastic Processes with Dynamic Normalizing Flows](https://mlsanity.com/paper/2002.10516/) (2020)
- [MCFlow: Monte Carlo Flow Models for Data Imputation](https://mlsanity.com/paper/2003.12628/) (2020)
- [CollaGAN : Collaborative GAN for Missing Image Data Imputation](https://mlsanity.com/paper/1901.09764/) (2019)
- [Missing Value Imputation on Multidimensional Time Series](https://mlsanity.com/paper/2103.01600/) (2021)
- [Longitudinal Variational Autoencoder](https://mlsanity.com/paper/2006.09763/) (2020)
- [NRTSI: Non-Recurrent Time Series Imputation](https://mlsanity.com/paper/2102.03340/) (2021)
- [CDSA: Cross-Dimensional Self-Attention for Multivariate, Geo-tagged Time Series Imputation](https://mlsanity.com/paper/1905.09904/) (2019)

## Built on by (most cited first, 30 of 169)

- [Diffusion-based Time Series Imputation and Forecasting with Structured State Space Models](https://mlsanity.com/paper/2208.09399/) (2022)
- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [PriSTI: A Conditional Diffusion Framework for Spatiotemporal Imputation](https://mlsanity.com/paper/2302.09746/) (2023)
- [Learning to Reconstruct Missing Data from Spatiotemporal Graphs with Sparse Observations](https://mlsanity.com/paper/2205.13479/) (2022)
- [Deep Learning for Multivariate Time Series Imputation: A Survey](https://mlsanity.com/paper/2402.04059/) (2024)
- [ImputeFormer: Low Rankness-Induced Transformers for Generalizable Spatiotemporal Imputation](https://mlsanity.com/paper/2312.01728/) (2023)
- [Time Weaver: A Conditional Time Series Generation Model](https://mlsanity.com/paper/2403.02682/) (2024)
- [Networked Time Series Imputation via Position-aware Graph Enhanced Variational Autoencoders](https://mlsanity.com/paper/2305.18612/) (2023)
- [Probabilistic Imputation for Time-series Classification with Missing Data](https://mlsanity.com/paper/2308.06738/) (2023)
- [GinAR: An End-To-End Multivariate Time Series Forecasting Model Suitable for Variable Missing](https://mlsanity.com/paper/2405.11333/) (2024)
- [Merlin: Multi-View Representation Learning for Robust Multivariate Time Series Forecasting with Unfixed Missing Rates](https://mlsanity.com/paper/2506.12459/) (2025)
- [SSD-TS: Exploring the Potential of Linear State Space Models for Diffusion Models in Time Series Imputation](https://mlsanity.com/paper/2410.13338/) (2024)
- [Time Series Continuous Modeling for Imputation and Forecasting with Implicit Neural Representations](https://mlsanity.com/paper/2306.05880/) (2023)
- [Interpretation of Time-Series Deep Models: A Survey](https://mlsanity.com/paper/2305.14582/) (2023)
- [Zero-shot Imputation with Foundation Inference Models for Dynamical Systems](https://mlsanity.com/paper/2402.07594/) (2024)
- [MTSCI: A Conditional Diffusion Model for Multivariate Time Series Consistent Imputation](https://mlsanity.com/paper/2408.05740/) (2024)
- [BayOTIDE: Bayesian Online Multivariate Time series Imputation with functional decomposition](https://mlsanity.com/paper/2308.14906/) (2023)
- [Self-Supervised Learning of Time Series Representation via Diffusion Process and Imputation-Interpolation-Forecasting Mask](https://mlsanity.com/paper/2405.05959/) (2024)
- [TimeCHEAT: A Channel Harmony Strategy for Irregularly Sampled Multivariate Time Series Analysis](https://mlsanity.com/paper/2412.12886/) (2024)
- [DiffECG: A Versatile Probabilistic Diffusion Model for ECG Signals Synthesis](https://mlsanity.com/paper/2306.01875/) (2023)
- [Glocal Information Bottleneck for Time Series Imputation](https://mlsanity.com/paper/2510.04910/) (2025)
- [Graph-based Virtual Sensing from Sparse and Partial Multivariate Observations](https://mlsanity.com/paper/2402.12598/) (2024)
- [Higher-order Spatio-temporal Physics-incorporated Graph Neural Network for Multivariate Time Series Imputation](https://mlsanity.com/paper/2405.10995/) (2024)
- [Pedestrian Trajectory Prediction with Missing Data: Datasets, Imputation, and Benchmarking](https://mlsanity.com/paper/2411.00174/) (2024)
- [LSCD: Lomb-Scargle Conditioned Diffusion for Time series Imputation](https://mlsanity.com/paper/2506.17039/) (2025)
- [sasdim: self-adaptive noise scaling diffusion model for spatial time series imputation](https://mlsanity.com/paper/2309.01988/) (2023)
- [Missing Value Imputation for Multi-attribute Sensor Data Streams via Message Propagation (Extended Version)](https://mlsanity.com/paper/2311.07344/) (2023)
- [Multi-scale Restoration of Missing Data in Optical Time-series Images with Masked Spatial-Temporal Attention Network](https://mlsanity.com/paper/2406.13358/) (2024)
- [Revisiting Multivariate Time Series Forecasting with Missing Values](https://mlsanity.com/paper/2509.23494/) (2025)
- [Correlated Attention in Transformers for Multivariate Time Series](https://mlsanity.com/paper/2311.11959/) (2023)

## Abstract (arXiv)

Missing data in time series is a pervasive problem that puts obstacles in the way of advanced analysis. A popular solution is imputation, where the fundamental challenge is to determine what values should be filled in. This paper proposes SAITS, a novel method based on the self-attention mechanism for missing value imputation in multivariate time series. Trained by a joint-optimization approach, SAITS learns missing values from a weighted combination of two diagonally-masked self-attention (DMSA) blocks. DMSA explicitly captures both the temporal dependencies and feature correlations between time steps, which improves imputation accuracy and training speed. Meanwhile, the weighted-combination design enables SAITS to dynamically assign weights to the learned representations from two DMSA blocks according to the attention map and the missingness information. Extensive experiments quantitatively and qualitatively demonstrate that SAITS outperforms the state-of-the-art methods on the time-series imputation task efficiently and reveal SAITS' potential to improve the learning performance of pattern recognition models on incomplete time-series data from the real world. The code is open source on GitHub at https://github.com/WenjieDu/SAITS.