mlsanityData to 7 Oct 2026

One Fits All:Power General Time Series Analysis by Pretrained LM

arXiv 2302.11939 ↗First posted 23 Feb 2023cs.LG cs.AIcited by 615

Not on the map. Why?

Cited by 26 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Although we have witnessed great success of pre-trained models in natural language processing (NLP) and computer vision (CV), limited progress has been made for general time series analysis. Unlike NLP and CV where a unified model can be used to perform different tasks, specially designed approach still dominates in each time series analysis task such as classification, anomaly detection, forecasting, and few-shot learning. The main challenge that blocks the development of pre-trained model for time series analysis is the lack of a large amount of data for training. In this work, we address this challenge by leveraging language or CV models, pre-trained from billions of tokens, for time series analysis. Specifically, we refrain from altering the self-attention and feedforward layers of the residual blocks in the pre-trained language or image model. This model, known as the Frozen Pretrained Transformer (FPT), is evaluated through fine-tuning on all major types of tasks involving time series. Our results demonstrate that pre-trained models on natural language or images can lead to a comparable or state-of-the-art performance in all main time series analysis tasks, as illustrated in Figure 1. We also found both theoretically and empirically that the self-attention module behaviors similarly to principle component analysis (PCA), an observation that helps explains how transformer bridges the domain gap and a crucial step towards understanding the universality of a pre-trained transformer.The code is publicly available at https://github.com/DAMO-DI-ML/One_Fits_All.

Key facts. Cited by 615 papers in the map (data through 7 Oct 2026). Builds on 45 of its 76 references that are in the map. First posted on arXiv 23 Feb 2023.

Builds on

45 of 76 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  8. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  9. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  10. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  11. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  12. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
33 more
  1. 2020Reformer: The Efficient Transformercited by 1,658
  2. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  3. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  4. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  5. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  6. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
  7. 2022In-context Learning and Induction Headscited by 930
  8. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
  9. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 748
  10. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 744
  11. 2018Deep learning for time series classification: a reviewcited by 575
  12. 2022Transformers in Time Series: A Surveycited by 489
  13. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 424
  14. 2018Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholdingcited by 407
  15. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 387
  16. 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 385
  17. 2019ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernelscited by 336
  18. 2018The UEA multivariate time series classification archive, 2018cited by 330
  19. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 273
  20. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 267
  21. 2021Global Filter Networks for Image Classificationcited by 257
  22. 2021Monash Time Series Forecasting Archivecited by 256
  23. 2023Looped Transformers as Programmable Computerscited by 224
  24. 2020The Lipschitz Constant of Self-Attentioncited by 171
  25. 2022Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structurescited by 158
  26. 2022N-HiTS: Neural Hierarchical Interpolation for Time Series Forecastingcited by 146
  27. 2022ETSformer: Exponential Smoothing Transformers for Time-series Forecastingcited by 146
  28. 2021Voice2Series: Reprogramming Acoustic Models for Time Series Classificationcited by 91
  29. 2020O(n) Connections are Expressive Enough: Universal Approximability of Sparse Transformerscited by 82
  30. 2020Meta-learning framework with applications to zero-shot time-series forecastingcited by 70
  31. 2021KVT: k-NN Attention for Boosting Vision Transformerscited by 64
  32. 2020RobustTAD: Robust Time Series Anomaly Detection via Decomposition and Convolutional Neural Networkscited by 54
  33. 2021On the Optimal Memorization Power of ReLU Neural Networkscited by 40

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 21; 2024: 198; 2025: 232; 2026: 164.

615 papers.

Sort:

By area:

  1. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  2. 2023A decoder-only foundation model for time-series forecastingcited by 650
  3. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  4. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  5. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
  6. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
  7. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 198
  8. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
  9. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 170
  10. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 165
  11. 2024Are Language Models Actually Useful for Time Series Forecasting?cited by 163
  12. 2023UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecastingcited by 139
38 more
  1. 2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasterscited by 127
  2. 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 120
  3. 2024Large Language Models for Time Series: A Surveycited by 113
  4. 2024TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysiscited by 113
  5. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  6. 2024CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuningcited by 105
  7. 2023Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecastingcited by 102
  8. 2024CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patternscited by 102
  9. 2024TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignmentcited by 97
  10. 2024ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoningcited by 95
  11. 2024UniTS: A Unified Multi-Task Time Series Modelcited by 93
  12. 2024From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflectioncited by 93
  13. 2024ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Datacited by 92
  14. 2024Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Expertscited by 88
  15. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 87
  16. 2024UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Predictioncited by 86
  17. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
  18. 2024Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunitiescited by 83
  19. 2024Spatial-Temporal Large Language Model for Traffic Predictioncited by 81
  20. 2024UrbanGPT: Spatio-Temporal Large Language Modelscited by 81
  21. 2024Language Models Still Struggle to Zero-shot Reason about Time Seriescited by 81
  22. 2024TSLANet: Rethinking Transformers for Time Series Representation Learningcited by 71
  23. 2024AutoTimes: Autoregressive Time Series Forecasters via Large Language Modelscited by 63
  24. 2024Empowering Time Series Analysis with Large Language Models: A Surveycited by 60
  25. 2025Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecastingcited by 59
  26. 2025A Comprehensive Survey of Deep Learning for Multivariate Time Series Forecasting: A Channel Strategy Perspectivecited by 58
  27. 2025TAB: Unified Benchmarking of Time Series Anomaly Detection Methodscited by 58
  28. 2024Deep Learning for Multivariate Time Series Imputation: A Surveycited by 57
  29. 2025This Time is Different: An Observability Perspective on Time Series Foundation Modelscited by 57
  30. 2024LSTPrompt: Large Language Models as Zero-Shot Time Series Forecasters by Long-Short-Term Promptingcited by 56
  31. 2023CARD: Channel Aligned Robust Blend Transformer for Time Series Forecastingcited by 55
  32. 2023A Survey on Time-Series Pre-Trained Modelscited by 52
  33. 2024EEGFormer: Towards Transferable and Interpretable Large-Scale EEG Foundation Modelcited by 51
  34. 2024Position: What Can Large Language Models Tell Us about Time Series Analysiscited by 47
  35. 2024TimeSeriesExam: A time series understanding examcited by 47
  36. 2025DBLoss: Decomposition-based Loss Function for Time Series Forecastingcited by 44
  37. 2025TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loopcited by 39
  38. 2024Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmarkcited by 38

50 most cited of 615 All 615 (JSON)