mlsanityData to 14 Oct 2026

ImputeFormer: Low Rankness-Induced Transformers for Generalizable Spatiotemporal Imputation

arXiv 2312.01728 ↗First posted 4 Dec 2023cs.LGcited by 50

Not on the map. Why?

Abstract (arXiv)

Missing data is a pervasive issue in both scientific and engineering tasks, especially for the modeling of spatiotemporal data. This problem attracts many studies to contribute to data-driven solutions. Existing imputation solutions mainly include low-rank models and deep learning models. The former assumes general structural priors but has limited model capacity. The latter possesses salient features of expressivity but lacks prior knowledge of the underlying spatiotemporal structures. Leveraging the strengths of both two paradigms, we demonstrate a low rankness-induced Transformer to achieve a balance between strong inductive bias and high model expressivity. The exploitation of the inherent structures of spatiotemporal data enables our model to learn balanced signal-noise representations, making it generalizable for a variety of imputation problems. We demonstrate its superiority in terms of accuracy, efficiency, and versatility in heterogeneous datasets, including traffic flow, solar energy, smart meters, and air quality. Promising empirical results provide strong conviction that incorporating time series primitives, such as low-rankness, can substantially facilitate the development of a generalizable model to approach a wide range of spatiotemporal imputation problems.

Key facts. Cited by 50 papers in the map (data through 14 Oct 2026). Builds on 43 of its 71 references that are in the map. First posted on arXiv 4 Dec 2023.

Builds on

43 of 71 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  3. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
  4. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  5. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  6. 2019Graph WaveNet for Deep Spatial-Temporal Graph Modelingcited by 829
  7. 2023A decoder-only foundation model for time-series forecastingcited by 692
  8. 2021CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputationcited by 582
  9. 2020Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networkscited by 566
  10. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 554
  11. 2018Efficient Attention: Attention with Linear Complexitiescited by 368
  12. 2018GAIN: Missing Data Imputation using Generative Adversarial Netscited by 352
31 more
  1. 2018BRITS: Bidirectional Recurrent Imputation for Time Seriescited by 290
  2. 2022Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistencycited by 250
  3. 2022SAITS: Self-Attention-based Imputation for Time Seriescited by 169
  4. 2023TimeGPT-1cited by 158
  5. 2022Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series Forecastingcited by 151
  6. 2023A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detectioncited by 145
  7. 2022Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecastingcited by 117
  8. 2023SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modelingcited by 115
  9. 2021Filling the G_ap_s: Multivariate Time Series Imputation by Graph Neural Networkscited by 95
  10. 2020Inductive Graph Neural Networks for Spatiotemporal Krigingcited by 91
  11. 2020Low-Rank Bottleneck in Multi-head Attention Modelscited by 89
  12. 2023PriSTI: A Conditional Diffusion Framework for Spatiotemporal Imputationcited by 66
  13. 2022Learning to Reconstruct Missing Data from Spatiotemporal Graphs with Sparse Observationscited by 64
  14. 2023Ti-MAE: Self-Supervised Masked Time Series Autoencoderscited by 63
  15. 2019NAOMI: Non-Autoregressive Multiresolution Sequence Imputationcited by 59
  16. 2019Bayesian Temporal Factorization for Multidimensional Time Series Predictioncited by 47
  17. 2019Physics-guided Convolutional Neural Network (PhyCNN) for Data-driven Seismic Response Modelingcited by 40
  18. 2023Taming Local Effects in Graph-based Spatiotemporal Forecastingcited by 37
  19. 2021Low-Rank Autoregressive Tensor Completion for Spatiotemporal Traffic Data Imputationcited by 35
  20. 2019Missing Not at Random in Matrix Completion: The Effectiveness of Estimating Missingness Probabilities Under a Low Nuclear Norm Assumptioncited by 29
  21. 2020A Nonconvex Low-Rank Tensor Completion Model for Spatiotemporal Traffic Data Imputationcited by 27
  22. 2021NRTSI: Non-Recurrent Time Series Imputationcited by 25
  23. 2019CDSA: Cross-Dimensional Self-Attention for Multivariate, Geo-tagged Time Series Imputationcited by 24
  24. 2021Low-Rank Hankel Tensor Completion for Traffic Speed Estimationcited by 22
  25. 2022Laplacian Convolutional Representation for Traffic Time Series Imputationcited by 18
  26. 2022Correlating sparse sensing for large-scale traffic speed estimation: A Laplacian-enhanced low-rank tensor kriging approachcited by 17
  27. 2019Recovery of Future Data via Convolution Nuclear Norm Minimizationcited by 12
  28. 2022Truncated tensor Schatten p-norm based approach for spatiotemporal traffic data imputation with complicated missing patternscited by 11
  29. 2021Time Series Forecasting via Learning Convolutionally Low-Rank Modelscited by 10
  30. 2021Dynamic Spatiotemporal Graph Convolutional Neural Networks for Traffic Data Imputation with Complex Missing Patternscited by 9
  31. 2023Towards better traffic volume estimation: Jointly addressing the underdetermination and nonequilibrium problems with correlation-adaptive GNNscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 13; 2025: 21; 2026: 16.

50 papers.

Sort:

By area:

  1. 2024Deep Learning for Multivariate Time Series Imputation: A Surveycited by 57
  2. 2025Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmapcited by 26
  3. 2024SSD-TS: Exploring the Potential of Linear State Space Models for Diffusion Models in Time Series Imputationcited by 18
  4. 2024Geolocation Representation from Large Language Models are Generic Enhancers for Spatio-Temporal Learningcited by 15
  5. 2024Channel-Aware Low-Rank Adaptation in Time Series Forecastingcited by 12
  6. 2024Expand and Compress: Exploring Tuning Principles for Continual Spatio-Temporal Graph Forecastingcited by 10
  7. 2024Diffusion Transformers as Open-World Spatiotemporal Foundation Modelscited by 10
  8. 2025Glocal Information Bottleneck for Time Series Imputationcited by 9
  9. 2024Spatiotemporal Implicit Neural Representation as a Generalized Traffic Data Learnercited by 7
  10. 2025Revisiting Multivariate Time Series Forecasting with Missing Valuescited by 7
  11. 2024Joint Estimation and Prediction of City-wide Delivery Demand: A Large Language Model Empowered Graph-based Learning Approachcited by 6
  12. 2024Task-oriented Time Series Imputation Evaluation via Generalized Representerscited by 6
38 more
  1. 2025FreshRetailNet-50K: A Stockout-Annotated Censored Demand Dataset for Latent Demand Recovery and Forecasting in Fresh Retailcited by 4
  2. 2025Decoupling Spatio-Temporal Prediction: When Lightweight Large Models Meet Adaptive Hypergraphscited by 4
  3. 2024Trajectory Imputation in Multi-Agent Sports with Derivative-Accumulating Self-Ensemblecited by 3
  4. 2025ImputeINR: Time Series Imputation via Implicit Neural Representations for Disease Diagnosis with Missing Datacited by 3
  5. 2025MH-GIN: Multi-scale Heterogeneous Graph-based Imputation Network for AIS Data (Extended Version)cited by 3
  6. 2024Latent Space Score-based Diffusion Model for Probabilistic Multivariate Time Series Imputationcited by 2
  7. 2025STAMImputer: Spatio-Temporal Attention MoE for Traffic Data Imputationcited by 2
  8. 2025FADTI: Fourier and Attention Driven Diffusion for Multivariate Time Series Imputationcited by 2
  9. 2026T1: One-to-One Channel-Head Binding for Multivariate Time-Series Imputationcited by 2
  10. 2024High-Dimensional Fault Tolerance Testing of Highly Automated Vehicles Based on Low-Rank Modelscited by 1
  11. 2025Causal View of Time Series Imputation: Some Identification Results on Missing Mechanismcited by 1
  12. 2025Predicting Large-scale Urban Network Dynamics with Energy-informed Graph Neural Diffusioncited by 1
  13. 2025FieldFormer: Locality-Aware Transformers for Spatio-Temporal Modeling on Sparse Sensor Networkscited by 1
  14. 2025RINS-T: Robust Implicit Neural Solvers for Time Series Linear Inverse Problemscited by 1
  15. 2025Are Time-Indexed Foundation Models the Future of Time Series Imputation?cited by 1
  16. 2026Spatial-Temporal Feedback Diffusion Guidance for Controlled Traffic Imputationcited by 1
  17. 2026UrbanFM: Scaling Urban Spatio-Temporal Foundation Modelscited by 1
  18. 2026SPLICE: Latent Diffusion over JEPA Embeddings for Conformal Time-Series Inpaintingcited by 1
  19. 2026HELIX: Hybrid Encoding with Learnable Identity and Cross-dimensional Synthesis for Time Series Imputationcited by 1
  20. 2026Existence Precedes Value: Joint Modeling of Observational Existence and Evolving States in Time Series Forecastingcited by 1
  21. 2024A Survey and Benchmarking of Spatial-Temporal Traffic Data Imputation Modelscited by 0
  22. 2025Collaborative Imputation of Urban Time Series through Cross-city Meta-learningcited by 0
  23. 2025MNT-TNN: Spatiotemporal Traffic Data Imputation via Compact Multimode Nonlinear Transform-based Tensor Nuclear Normcited by 0
  24. 2025Comparative Analysis of Modern Machine Learning Models for Retail Sales Forecastingcited by 0
  25. 2025GRAM-MAMBA: Holistic Feature Alignment for Wireless Perception with Adaptive Low-Rank Compensationcited by 0
  26. 2025Physics-informed deep operator network for traffic state estimationcited by 0
  27. 2025GeoMAE: Masking Representation Learning for Spatio-Temporal Graph Forecasting with Missing Valuescited by 0
  28. 2025AdaSTI: Conditional Diffusion Models with Adaptive Dependency Modeling for Spatio-Temporal Imputationcited by 0
  29. 2026Multivariate Time Series Data Imputation via Distributionally Robust Regularizationcited by 0
  30. 2026Uniform Inductive Spatio-Temporal Krigingcited by 0
  31. 2026DiSGMM: A Method for Time-varying Microscopic Weight Completion on Road Networkscited by 0
  32. 2026MBDiff: Multi-view Behavior-aware Diffusion Model for Probabilistic Utility Data Imputationcited by 0
  33. 2026GLAIM: Learning Global and Local Adaptive Inter-Variable Dependency for Multivariate Time Series Imputationcited by 0
  34. 2026Discretizing Continuous Time Series for Imputation with Masked Diffusion Trainingcited by 0
  35. 2026RDDMPI: Residual Denoising Diffusion Model for Probabilistic Multivariate Time Series Imputationcited by 0
  36. 2026Context-Aware Causal Gaze Forecasting for Human-Vehicle Interaction During In-Cabin Tracking Dropoutscited by 0
  37. 2026When Does Adversarial Refinement Help? A Negative Result and Open Problem in Adapting R3GAN to Time Series Imputationcited by 0
  38. 2026ProCTI: Prototype-Refined Global Conditioning for Diffusion-Based Time Series Imputationcited by 0