mlsanityData to 7 Oct 2026

TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series

arXiv 2308.08241 ↗First posted 16 Aug 2023cs.CL cs.AIcited by 165

Not on the map. Why?

Abstract (arXiv)

This work summarizes two ways to accomplish Time-Series (TS) tasks in today's Large Language Model (LLM) context: LLM-for-TS (model-centric) designs and trains a fundamental large model, or fine-tunes a pre-trained LLM for TS data; TS-for-LLM (data-centric) converts TS into a model-friendly representation to enable the pre-trained LLM to handle TS data. Given the lack of data, limited resources, semantic context requirements, and so on, this work focuses on TS-for-LLM, where we aim to activate LLM's ability for TS data by designing a TS embedding method suitable for LLM. The proposed method is named TEST. It first tokenizes TS, builds an encoder to embed TS via instance-wise, feature-wise, and text-prototype-aligned contrast, where the TS embedding space is aligned to LLM embedding layer space, then creates soft prompts to make LLM more open to that embeddings, and finally implements TS tasks using the frozen LLM. We also demonstrate the feasibility of TS-for-LLM through theory and experiments. Experiments are carried out on TS classification, forecasting, and representation tasks using eight frozen LLMs with various structures and sizes. The results show that the pre-trained LLM with TEST strategy can achieve better or comparable performance than today's SOTA TS models and offer benefits for few-shot and generalization. By treating LLM as the pattern machine, TEST can endow LLM's ability to process TS data without compromising language ability. We hope that this study will serve as a foundation for future work to support TS+LLM progress.

Key facts. Cited by 165 papers in the map (data through 7 Oct 2026). Builds on 54 of its 74 references that are in the map. First posted on arXiv 16 Aug 2023.

Builds on

54 of 74 references.

References · 54

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  9. 2023A Survey of Large Language Modelscited by 2,537
  10. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  11. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  12. 2020Reformer: The Efficient Transformercited by 1,658
38 more
  1. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  2. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  3. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  4. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  5. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
  6. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  7. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 744
  8. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 636
  9. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  10. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 463
  11. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
  12. 2018The UCR Time Series Archivecited by 421
  13. 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 385
  14. 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 342
  15. 2018The UEA multivariate time series classification archive, 2018cited by 330
  16. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 267
  17. 2020Contrastive Clusteringcited by 264
  18. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 223
  19. 2023Large Language Models as General Pattern Machinescited by 196
  20. 2020MINIROCKET: A Very Fast (Almost) Deterministic Transform for Time Series Classificationcited by 186
  21. 2022PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecastingcited by 170
  22. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
  23. 2022ETSformer: Exponential Smoothing Transformers for Time-series Forecastingcited by 146
  24. 2022CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecastingcited by 143
  25. 2018Multivariate LSTM-FCNs for Time Series Classificationcited by 120
  26. 2023Large Language Models are Few-Shot Health Learnerscited by 115
  27. 2023SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modelingcited by 112
  28. 2022Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusioncited by 96
  29. 2021Supporting Clustering with Contrastive Learningcited by 90
  30. 2023Frozen Language Model Helps ECG Zero-Shot Learningcited by 62
  31. 2021Learning Representation for Clustering via Prototype Scattering and Positive Samplingcited by 58
  32. 2022Mixing Up Contrastive Learning: Self-Supervised Representation Learning for Time Seriescited by 54
  33. 2017Multivariate Time Series Classification with WEASEL+MUSEcited by 53
  34. 2023A Survey on Time-Series Pre-Trained Modelscited by 52
  35. 2020A Review of Deep Learning Methods for Irregularly Sampled Medical Time Series Datacited by 29
  36. 2022MHCCL: Masked Hierarchical Cluster-Wise Contrastive Learning for Multivariate Time Seriescited by 28
  37. 2020Clustering based Contrastive Learning for Improving Face Representationscited by 21
  38. 2018Unsupervised Hyperalignment for Multilingual Word Embeddingscited by 17

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 56; 2025: 64; 2026: 38.

165 papers.

Sort:

By area:

  1. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
  2. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 170
  3. 2024Are Language Models Actually Useful for Time Series Forecasting?cited by 163
  4. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
  5. 2024Large Language Models for Time Series: A Surveycited by 113
  6. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  7. 2024CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuningcited by 105
  8. 2024TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignmentcited by 97
  9. 2024UniTS: A Unified Multi-Task Time Series Modelcited by 93
  10. 2024From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflectioncited by 93
  11. 2024Spatial-Temporal Large Language Model for Traffic Predictioncited by 81
  12. 2024AutoTimes: Autoregressive Time Series Forecasters via Large Language Modelscited by 63
38 more
  1. 2024Empowering Time Series Analysis with Large Language Models: A Surveycited by 60
  2. 2024LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddingscited by 60
  3. 2024Time Series Forecasting with LLMs: Understanding and Enhancing Model Capabilitiescited by 48
  4. 2024Position: What Can Large Language Models Tell Us about Time Series Analysiscited by 47
  5. 2023LLM4DyG: Can Large Language Models Solve Spatial-Temporal Problems on Dynamic Graphs?cited by 40
  6. 2025TS-RAG: Retrieval-Augmented Generation based Time Series Foundation Models are Stronger Zero-Shot Forecastercited by 36
  7. 2023GATGPT: A Pre-trained Large Language Model with Graph Attention Network for Spatiotemporal Imputationcited by 35
  8. 2024SensorLLM: Aligning Large Language Models with Motion Sensors for Human Activity Recognitioncited by 35
  9. 2025Context-Alignment: Activating and Enhancing LLM Capabilities in Time Seriescited by 34
  10. 2024TimeDiT: General-purpose Diffusion Transformers for Time Series Foundation Modelcited by 33
  11. 2025TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agentscited by 33
  12. 2024Universal Time-Series Representation Learning: A Surveycited by 28
  13. 2024Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detectioncited by 26
  14. 2024Multi-Patch Prediction: Adapting LLMs for Time Series Representation Learningcited by 25
  15. 2024Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Modelscited by 23
  16. 2024TPLLM: A Traffic Prediction Framework Based on Pretrained Large Language Modelscited by 23
  17. 2024Empowering Time Series Analysis with Foundation Models: A Comprehensive Surveycited by 23
  18. 2024Specialized Foundation Models Struggle to Beat Supervised Baselinescited by 23
  19. 2024UniMTS: Unified Pre-training for Motion Time Seriescited by 22
  20. 2024Decoding Time Series with LLMs: A Multi-Agent Framework for Cross-Domain Annotationcited by 21
  21. 2024Mobility-LLM: Learning Visiting Intentions and Travel Preferences from Human Mobility Data with Large Language Modelscited by 21
  22. 2025How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlookcited by 20
  23. 2025Efficient Multivariate Time Series Forecasting via Calibrated Language Models with Privileged Knowledge Distillationcited by 20
  24. 2025True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statisticscited by 20
  25. 2024xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memoriescited by 18
  26. 2025A Survey of AIOps in the Era of Large Language Modelscited by 18
  27. 2024S2IP-LLM: Semantic Space Informed Prompt Learning with LLM for Time Series Forecastingcited by 17
  28. 2025TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learningcited by 16
  29. 2024Geolocation Representation from Large Language Models are Generic Enhancers for Spatio-Temporal Learningcited by 15
  30. 2024Rethinking the Power of Timestamps for Robust Time Series Forecasting: A Global-Local Fusion Perspectivecited by 13
  31. 2025From Values to Tokens: An LLM-Driven Framework for Context-aware Time Series Forecasting via Symbolic Discretizationcited by 13
  32. 2025A Survey on Mathematical Reasoning and Optimization with Large Language Modelscited by 12
  33. 2024DynLLM: When Large Language Models Meet Dynamic Graph Recommendationcited by 11
  34. 2025Towards Cross-Modality Modeling for Time Series Analytics: A Survey in the LLM Eracited by 11
  35. 2025BEDTime: A Unified Benchmark for Automatically Describing Time Seriescited by 11
  36. 2024Self-Supervised Learning of Time Series Representation via Diffusion Process and Imputation-Interpolation-Forecasting Maskcited by 10
  37. 2024Unleashing The Power of Pre-Trained Language Models for Irregularly Sampled Time Seriescited by 10
  38. 2025Deep Learning for Time Series Forecasting: A Surveycited by 10

50 most cited of 165 All 165 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025TsLLM: Augmenting LLMs for General Time Series Understanding and Prediction
  2. 2025Adapting Large Language Models for Time Series Modeling via a Novel Parameter-efficient Adaptation Method
  3. 2025When LLM Meets Time Series: Can LLMs Perform Multi-Step Time Series Reasoning and Inference
  4. 2025Adapting LLMs to Time Series Forecasting via Temporal Heterogeneity Modeling and Representation Alignment
  5. 2024NeST: Neighborhood-aware semantic alignment and temporal modulation for LLM based time series forecasting
  6. 2026Rethinking the Role of LLMs in Time Series Forecasting
  7. 2026Thoth: Mid-Training Bridges LLMs to Time Series Understanding
  8. 2024A Picture is Worth A Thousand Numbers: Enabling LLMs Reason about Time Series via Visualization