mlsanityData to 7 Oct 2026

Time Series as Images: Vision Transformer for Irregularly Sampled Time Series

arXiv 2303.12799 ↗First posted 1 Mar 2023cs.LG cs.AI cs.CVcited by 79

Not on the map. Why?

Abstract (arXiv)

Irregularly sampled time series are increasingly prevalent, particularly in medical domains. While various specialized methods have been developed to handle these irregularities, effectively modeling their complex dynamics and pronounced sparsity remains a challenge. This paper introduces a novel perspective by converting irregularly sampled time series into line graph images, then utilizing powerful pre-trained vision transformers for time series classification in the same way as image classification. This method not only largely simplifies specialized algorithm designs but also presents the potential to serve as a universal framework for time series modeling. Remarkably, despite its simplicity, our approach outperforms state-of-the-art specialized algorithms on several popular healthcare and human activity datasets. Especially in the rigorous leave-sensors-out setting where a portion of variables is omitted during testing, our method exhibits strong robustness against varying degrees of missing observations, achieving an impressive improvement of 42.8% in absolute F1 score points over leading specialized baselines even with half the variables masked. Code and data are available at https://github.com/Leezekun/ViTST

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). Builds on 45 of its 64 references that are in the map. First posted on arXiv 1 Mar 2023.

Builds on

45 of 64 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  8. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  9. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  11. 2017Inductive Representation Learning on Large Graphscited by 6,284
  12. 2016Layer Normalizationcited by 5,579
33 more
  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  2. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  3. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  4. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  5. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  6. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  7. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  8. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  9. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  10. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  11. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  12. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 606
  13. 2020Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networkscited by 563
  14. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 548
  15. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 504
  16. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
  17. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 387
  18. 2019ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernelscited by 336
  19. 2018The UEA multivariate time series classification archive, 2018cited by 330
  20. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 267
  21. 2021Multi-Time Attention Networks for Irregularly Sampled Time Seriescited by 234
  22. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  23. 2019Set Functions for Time Seriescited by 137
  24. 2021Graph-Guided Network for Irregularly Sampled Multivariate Time Seriescited by 110
  25. 2019Interpolation-Prediction Networks for Irregularly Sampled Time Seriescited by 89
  26. 2015Imaging Time-Series to Improve Classification and Imputationcited by 85
  27. 2016Modeling Missing Data in Clinical Time Series with RNNscited by 80
  28. 2017Classification of Time-Series Images Using Deep Convolutional Neural Networkscited by 38
  29. 2022Improving Medical Predictions by Irregular Multimodal Electronic Health Records Modelingcited by 33
  30. 2021Dynamic Gaussian Mixture based Deep Generative Model For Robust Forecasting on Sparse Multivariate Time Seriescited by 30
  31. 2021Can An Image Classifier Suffice For Action Recognition?cited by 20
  32. 2020Visual Time Series Forecasting: An Image-driven Approachcited by 13
  33. 2015Spatially Encoding Temporal Correlations to Classify Temporal Data Using Convolutional Neural Networkscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 15; 2025: 39; 2026: 24.

79 papers.

Sort:

By area:

  1. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 87
  2. 2024Language Models Still Struggle to Zero-shot Reason about Time Seriescited by 81
  3. 2024Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classificationcited by 53
  4. 2024Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Seriescited by 42
  5. 2025TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loopcited by 39
  6. 2025Representation Learning for Tabular Data: A Comprehensive Surveycited by 36
  7. 2025Harnessing Vision Models for Time Series Analysis: A Surveycited by 26
  8. 2025Multi-modal Time Series Analysis: A Tutorial and Surveycited by 24
  9. 2024Empowering Time Series Analysis with Foundation Models: A Comprehensive Surveycited by 23
  10. 2024Trajectory Flow Matching with Applications to Clinical Time Series Modelingcited by 21
  11. 2024See it, Think it, Sorted: Large Multimodal Models are Few-shot Time Series Anomaly Analyzerscited by 20
  12. 2025How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlookcited by 20
38 more
  1. 2025CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Datacited by 12
  2. 2025Can Multimodal LLMs Perform Time Series Anomaly Detection?cited by 11
  3. 2025VisionTS++: Cross-Modal Time Series Foundation Model with Continual Pre-trained Vision Backbonescited by 11
  4. 2024TimeCHEAT: A Channel Harmony Strategy for Irregularly Sampled Multivariate Time Series Analysiscited by 10
  5. 2025Harnessing Vision-Language Models for Time Series Anomaly Detectioncited by 9
  6. 2026EEO-TFV: Escape-Explore Optimizer for Web-Scale Time-Series Forecasting and Vision Analysiscited by 9
  7. 2024TimEHR: Image-based Time Series Generation for Electronic Health Recordscited by 8
  8. 2025Multi-Modal View Enhanced Large Vision Models for Long-Term Time Series Forecastingcited by 8
  9. 2024LITE: Modeling Environmental Ecosystems with Multimodal Large Language Modelscited by 7
  10. 2025From Images to Signals: Are Large Vision Models Useful for Time Series Analysis?cited by 7
  11. 2025Teaching Time Series to See and Speak: Forecasting with Aligned Visual and Textual Perspectivescited by 7
  12. 2025A Diffusion Model for Regular Time Series Generation from Irregular Data with Completion and Maskingcited by 7
  13. 2025Unlocking Pretrained Vision Transformers for Time Series Classificationcited by 6
  14. 2025Unifying Prediction and Explanation in Time-Series Transformers via Shapley-based Pretrainingcited by 5
  15. 2025AimTS: Augmented Series and Image Contrastive Learning for Time Series Classificationcited by 5
  16. 2024MUSE-Net: Missingness-aware mUlti-branching Self-attention Encoder for Irregular Longitudinal Electronic Health Recordscited by 4
  17. 2024Irregularity-Informed Time Series Analysis: Adaptive Modelling of Spatial and Temporal Dynamicscited by 4
  18. 2025Rotary Masked Autoencoders are Versatile Learnerscited by 4
  19. 2025MLLM4TS: Leveraging Vision and Multimodal Language Models for General Time-Series Analysiscited by 4
  20. 2025TADACap: Time-series Adaptive Domain-Aware Captioningcited by 3
  21. 2025Unsupervised Time-Series Signal Analysis with Autoencoders and Vision Transformers: A Review of Architectures and Applicationscited by 3
  22. 2025MONAQ: Multi-Objective Neural Architecture Querying for Time-Series Analysis on Resource-Constrained Devicescited by 2
  23. 2025Signal, Image, or Symbolic: Exploring the Best Input Representation for Electrocardiogram-Language Models Through a Unified Frameworkcited by 2
  24. 2025IMTS is Worth Time × Channel Patches: Visual Masked Autoencoders for Irregular Multivariate Time Series Predictioncited by 2
  25. 2025MTM: A Multi-Scale Token Mixing Transformer for Irregular Multivariate Time Series Classificationcited by 2
  26. 2025Mind the Missing: Variable-Aware Representation Learning for Irregular EHR Time Series using Large Language Modelscited by 2
  27. 2026MILM: Large Language Models for Multimodal Irregular Time Series with Informative Samplingcited by 2
  28. 2023No Imputation Needed: A Switch Approach to Irregularly Sampled Time Seriescited by 1
  29. 2024LETS-C: Leveraging Text Embedding for Time Series Classificationcited by 1
  30. 2025Integrating Sequence and Image Modeling in Irregular Medical Time Series Through Self-Supervised Learningcited by 1
  31. 2025STaRFormer: Semi-Supervised Task-Informed Representation Learning via Dynamic Attention-Based Regional Masking for Sequential Datacited by 1
  32. 2025Towards Robust Real-World Multivariate Time Series Forecasting: A Unified Framework for Dependency, Asynchrony, and Missingnesscited by 1
  33. 2025UniCast: A Unified Framework for Instance-Conditioned Multimodal Time-Series Forecastingcited by 1
  34. 2025Beyond Observations: Reconstruction Error-Guided Irregularly Sampled Time Series Representation Learningcited by 1
  35. 2026VTBench: A Multimodal Framework for Time-Series Classification with Chart-Based Representationscited by 1
  36. 2026One-Step Graph-Structured Neural Flows for Irregular Multivariate Time Series Classificationcited by 1
  37. 2026Net-Ev2: A Generative Simulator for Network Event Evolutioncited by 1
  38. 2024MuSiCNet: A Gradual Coarse-to-Fine Framework for Irregularly Sampled Multivariate Time Series Analysiscited by 0

50 most cited of 79 All 79 (JSON)