mlsanityData to 7 Oct 2026

Unlocking Pretrained Vision Transformers for Time Series Classification

arXiv 2506.08641 ↗First posted 10 Jun 2025cs.LG cs.AI cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Adapting vision models for time series analysis is compelling, yet all existing approaches are falling short of dedicated time series foundation models (TSFMs) in classification. In this work, we propose Time Vision Transformer (TiViT), the first framework that successfully unlocks the representational power of frozen Vision Transformers (ViTs) pretrained on large-scale image datasets for time series classification. TiViT achieves state-of-the-art performance without any finetuning by utilizing the hidden representations of OpenCLIP models. We explore the structure of TiViT representations and find that intermediate ViT layers with high intrinsic dimension are the most effective for time series classification. Furthermore, we assess the alignment between TiViT and TSFM representation spaces and identify a strong complementarity, with additional performance gains achieved through feature concatenation. Finally, we unfreeze the ViT backbone of TiViT for continual pretraining and contrastive alignment with TSFMs on time series, enhancing the performance of lightweight TiViT variants. Our findings reveal a new direction for the domain and task adaptation of vision foundation models. Code is available at https://github.com/ExplainableML/TiViT.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 57 of its 79 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

57 of 79 references.

References · 57

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  8. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  9. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  10. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  11. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  12. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
38 more
  1. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  2. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
  3. 2020Quantifying Attention Flow in Transformerscited by 759
  4. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 744
  5. 2024Chronos: Learning the Language of Time Seriescited by 712
  6. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  7. 2023A decoder-only foundation model for time-series forecastingcited by 650
  8. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  9. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  10. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 463
  11. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
  12. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
  13. 2018The UCR Time Series Archivecited by 421
  14. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 387
  15. 2016Time Series Classification from Scratch with Deep Neural Networks: A Strong Baselinecited by 358
  16. 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 342
  17. 2018The UEA multivariate time series classification archive, 2018cited by 330
  18. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 267
  19. 2018Estimating the intrinsic dimension of datasets by a minimal neighborhood informationcited by 246
  20. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 223
  21. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 198
  22. 2022PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecastingcited by 170
  23. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 170
  24. 2020Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progresscited by 148
  25. 2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasterscited by 127
  26. 2023The geometry of hidden representations of large transformer modelscited by 108
  27. 2023Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecastingcited by 102
  28. 2016Multi-Scale Convolutional Neural Networks for Time Series Classificationcited by 99
  29. 2024UniTS: A Unified Multi-Task Time Series Modelcited by 93
  30. 2024Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Expertscited by 88
  31. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 87
  32. 2023Time Series as Images: Vision Transformer for Irregularly Sampled Time Seriescited by 79
  33. 2023A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexitycited by 77
  34. 2022What do Vision Transformers Learn? A Visual Explorationcited by 75
  35. 2024SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attentioncited by 43
  36. 2022WOODS: Benchmarks for Out-of-Distribution Generalization in Time Seriescited by 29
  37. 2018Towards a universal neural network encoder for time seriescited by 28
  38. 2025Harnessing Vision Models for Time Series Analysis: A Surveycited by 26

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2025CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Datacited by 12
  2. 2026MantisV2: Closing the Zero-Shot Gap in Time Series Classification with Synthetic Data and Test-Time Strategiescited by 10
  3. 2026Prompting Underestimates LLM Capability for Time Series Classificationcited by 1
  4. 2026Rethinking Intrinsic Dimension Estimation in Neural Representationscited by 0
  5. 2026EvoTSC: Evolving Feature Learning Models for Time Series Classification via Genetic Programmingcited by 0
  6. 2026LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Seriescited by 0