mlsanityData to 7 Oct 2026

SAITS: Self-Attention-based Imputation for Time Series

arXiv 2202.08516 ↗First posted 17 Feb 2022cs.LGcited by 167

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Missing data in time series is a pervasive problem that puts obstacles in the way of advanced analysis. A popular solution is imputation, where the fundamental challenge is to determine what values should be filled in. This paper proposes SAITS, a novel method based on the self-attention mechanism for missing value imputation in multivariate time series. Trained by a joint-optimization approach, SAITS learns missing values from a weighted combination of two diagonally-masked self-attention (DMSA) blocks. DMSA explicitly captures both the temporal dependencies and feature correlations between time steps, which improves imputation accuracy and training speed. Meanwhile, the weighted-combination design enables SAITS to dynamically assign weights to the learned representations from two DMSA blocks according to the attention map and the missingness information. Extensive experiments quantitatively and qualitatively demonstrate that SAITS outperforms the state-of-the-art methods on the time-series imputation task efficiently and reveal SAITS' potential to improve the learning performance of pattern recognition models on incomplete time-series data from the real world. The code is open source on GitHub at https://github.com/WenjieDu/SAITS.

Key facts. Cited by 167 papers in the map (data through 7 Oct 2026). Builds on 34 of its 62 references that are in the map. First posted on arXiv 17 Feb 2022.

Builds on

34 of 62 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2016Layer Normalizationcited by 5,579
  7. 2016Improved Techniques for Training GANscited by 2,938
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  9. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  10. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  11. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
  12. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 548
22 more
  1. 2018GAIN: Missing Data Imputation using Generative Adversarial Netscited by 349
  2. 2018BRITS: Bidirectional Recurrent Imputation for Time Seriescited by 288
  3. 2019GP-VAE: Deep Probabilistic Time Series Imputationcited by 171
  4. 2018Handling Incomplete Heterogeneous Data using VAEscited by 148
  5. 2019Set Functions for Time Seriescited by 137
  6. 2017Estimating Missing Data in Temporal Data Streams Using Multi-directional Recurrent Neural Networkscited by 86
  7. 2019MisGAN: Learning from Incomplete Data with Generative Adversarial Networkscited by 74
  8. 2018MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcarecited by 72
  9. 2018Gaussian Process Prior Variational Autoencoderscited by 72
  10. 2017DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understandingcited by 71
  11. 2019NAOMI: Non-Autoregressive Multiresolution Sequence Imputationcited by 59
  12. 2020Modeling Continuous Stochastic Processes with Dynamic Normalizing Flowscited by 37
  13. 2020MCFlow: Monte Carlo Flow Models for Data Imputationcited by 37
  14. 2019CollaGAN : Collaborative GAN for Missing Image Data Imputationcited by 33
  15. 2021Missing Value Imputation on Multidimensional Time Seriescited by 33
  16. 2020Longitudinal Variational Autoencodercited by 25
  17. 2021NRTSI: Non-Recurrent Time Series Imputationcited by 25
  18. 2019CDSA: Cross-Dimensional Self-Attention for Multivariate, Geo-tagged Time Series Imputationcited by 24
  19. 2020Sparse Gaussian Process Variational Autoencoderscited by 21
  20. 2020Fast and Accurate Deep Bidirectional Language Representations for Unsupervised Learningcited by 6
  21. 2022Forecasting Loss of Signal in Optical Networks with Machine Learningcited by 2
  22. 2021MAIN: Multihead-Attention Imputation Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 23; 2024: 42; 2025: 51; 2026: 48.

167 papers.

Sort:

By area:

  1. 2022Diffusion-based Time Series Imputation and Forecasting with Structured State Space Modelscited by 195
  2. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  3. 2022Learning to Reconstruct Missing Data from Spatiotemporal Graphs with Sparse Observationscited by 64
  4. 2023PriSTI: A Conditional Diffusion Framework for Spatiotemporal Imputationcited by 62
  5. 2024Deep Learning for Multivariate Time Series Imputation: A Surveycited by 57
  6. 2023ImputeFormer: Low Rankness-Induced Transformers for Generalizable Spatiotemporal Imputationcited by 50
  7. 2024Time Weaver: A Conditional Time Series Generation Modelcited by 35
  8. 2023Networked Time Series Imputation via Position-aware Graph Enhanced Variational Autoencoderscited by 32
  9. 2023Probabilistic Imputation for Time-series Classification with Missing Datacited by 21
  10. 2024GinAR: An End-To-End Multivariate Time Series Forecasting Model Suitable for Variable Missingcited by 20
  11. 2025Merlin: Multi-View Representation Learning for Robust Multivariate Time Series Forecasting with Unfixed Missing Ratescited by 19
  12. 2024SSD-TS: Exploring the Potential of Linear State Space Models for Diffusion Models in Time Series Imputationcited by 18
38 more
  1. 2023Time Series Continuous Modeling for Imputation and Forecasting with Implicit Neural Representationscited by 16
  2. 2023Interpretation of Time-Series Deep Models: A Surveycited by 15
  3. 2024Zero-shot Imputation with Foundation Inference Models for Dynamical Systemscited by 13
  4. 2024MTSCI: A Conditional Diffusion Model for Multivariate Time Series Consistent Imputationcited by 13
  5. 2024Self-Supervised Learning of Time Series Representation via Diffusion Process and Imputation-Interpolation-Forecasting Maskcited by 10
  6. 2024TimeCHEAT: A Channel Harmony Strategy for Irregularly Sampled Multivariate Time Series Analysiscited by 10
  7. 2023DiffECG: A Versatile Probabilistic Diffusion Model for ECG Signals Synthesiscited by 9
  8. 2023BayOTIDE: Bayesian Online Multivariate Time series Imputation with functional decompositioncited by 9
  9. 2025Glocal Information Bottleneck for Time Series Imputationcited by 9
  10. 2024Graph-based Virtual Sensing from Sparse and Partial Multivariate Observationscited by 8
  11. 2024Higher-order Spatio-temporal Physics-incorporated Graph Neural Network for Multivariate Time Series Imputationcited by 8
  12. 2024Pedestrian Trajectory Prediction with Missing Data: Datasets, Imputation, and Benchmarkingcited by 8
  13. 2025LSCD: Lomb-Scargle Conditioned Diffusion for Time series Imputationcited by 8
  14. 2023sasdim: self-adaptive noise scaling diffusion model for spatial time series imputationcited by 7
  15. 2024Multi-scale Restoration of Missing Data in Optical Time-series Images with Masked Spatial-Temporal Attention Networkcited by 7
  16. 2025Revisiting Multivariate Time Series Forecasting with Missing Valuescited by 7
  17. 2023Missing Value Imputation for Multi-attribute Sensor Data Streams via Message Propagation (Extended Version)cited by 6
  18. 2023Correlated Attention in Transformers for Multivariate Time Seriescited by 6
  19. 2023Continuous-time Autoencoders for Regular and Irregular Time Series Imputationcited by 6
  20. 2024Score-CDM: Score-Weighted Convolutional Diffusion Model for Multivariate Time Series Imputationcited by 6
  21. 2024Beyond Random Missingness: Clinically Rethinking for Healthcare Time Series Imputationcited by 6
  22. 2024Joint Estimation and Prediction of City-wide Delivery Demand: A Large Language Model Empowered Graph-based Learning Approachcited by 6
  23. 2024Task-oriented Time Series Imputation Evaluation via Generalized Representerscited by 6
  24. 2025S4M: S4 for multivariate time series forecasting with Missing valuescited by 6
  25. 2024How Deep is your Guess? A Fresh Perspective on Deep Learning for Medical Time-Series Imputationcited by 5
  26. 2024Mining of Switching Sparse Networks for Missing Value Imputation in Multivariate Time Seriescited by 5
  27. 2025Self-attention-based Diffusion Model for Time-series Imputation in Partial Blackout Scenarioscited by 5
  28. 2023Deep Geometric Learning with Monotonicity Constraints for Alzheimer's Disease Progressioncited by 4
  29. 2023ADF & TransApp: A Transformer-Based Framework for Appliance Detection Using Smart Meter Consumption Seriescited by 4
  30. 2024Learnable Prompt as Pseudo-Imputation: Rethinking the Necessity of Traditional EHR Data Imputation in Downstream Clinical Predictioncited by 4
  31. 2024NuwaTS: a Foundation Model Mending Every Incomplete Time Seriescited by 4
  32. 2025FreshRetailNet-50K: A Stockout-Annotated Censored Demand Dataset for Latent Demand Recovery and Forecasting in Fresh Retailcited by 4
  33. 2025Cross-Domain Conditional Diffusion Models for Time Series Imputationcited by 4
  34. 2023Do Machine Learning Models Learn Statistical Rules Inferred from Data?cited by 3
  35. 2024Representation Learning for Wearable-Based Applications in the Case of Missing Datacited by 3
  36. 2024Temporally Multi-Scale Sparse Self-Attention for Physical Activity Data Imputationcited by 3
  37. 2024Trajectory Imputation in Multi-Agent Sports with Derivative-Accumulating Self-Ensemblecited by 3
  38. 2025Towards Explainable Fusion and Balanced Learning in Multimodal Sentiment Analysiscited by 3

50 most cited of 167 All 167 (JSON)