mlsanityData to 7 Oct 2026

Unleashing The Power of Pre-Trained Language Models for Irregularly Sampled Time Series

arXiv 2408.08328 ↗First posted 12 Aug 2024cs.AI cs.LG stat.APcited by 10

Not on the map. Why?

Abstract (arXiv)

Pre-trained Language Models (PLMs), such as ChatGPT, have significantly advanced the field of natural language processing. This progress has inspired a series of innovative studies that explore the adaptation of PLMs to time series analysis, intending to create a unified foundation model that addresses various time series analytical tasks. However, these efforts predominantly focus on Regularly Sampled Time Series (RSTS), neglecting the unique challenges posed by Irregularly Sampled Time Series (ISTS), which are characterized by uneven sampling intervals and prevalent missing data. To bridge this gap, this work takes the first step in exploring the potential of PLMs for ISTS analysis. We begin by investigating the effect of various methods for representing ISTS, aiming to maximize the efficacy of PLMs in the analysis. Furthermore, we propose a unified PLM-based framework, named ISTS-PLM, to address diverse ISTS analytical tasks. It integrates novel time-aware and variable-aware PLMs tailored to tackle the intractable intra- and inter-time series modeling in ISTS. Finally, extensive experiments on a comprehensive benchmark demonstrate that the ISTS-PLM, utilizing a structured and effective series-based representation for ISTS, consistently achieves state-of-the-art performance across various analytical tasks, such as classification, interpolation, extrapolation, few-shot and zero-shot learning scenarios, spanning scientific domains like healthcare, biomechanics, and climate science.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 26 of its 47 references that are in the map. First posted on arXiv 12 Aug 2024.

Builds on

26 of 47 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  3. 2018Neural Ordinary Differential Equationscited by 3,733
  4. 2023A Survey on Multimodal Large Language Modelscited by 898
  5. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  6. 2020Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networkscited by 563
  7. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 548
  8. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  9. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
  10. 2021Multi-Time Attention Networks for Irregularly Sampled Time Seriescited by 234
  11. 2019GRU-ODE-Bayes: Continuous modeling of sporadically-observed time seriescited by 230
  12. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 165
14 more
  1. 2021Graph-Guided Network for Irregularly Sampled Multivariate Time Seriescited by 110
  2. 2021Modeling Irregular Time Series with Continuous Recurrent Unitscited by 101
  3. 2021Neural Flows: Efficient Alternative to Neural ODEscited by 99
  4. 2016Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequencescited by 98
  5. 2019Interpolation-Prediction Networks for Irregularly Sampled Time Seriescited by 89
  6. 2016Modeling Missing Data in Clinical Time Series with RNNscited by 80
  7. 2024AutoTimes: Autoregressive Time Series Forecasters via Large Language Modelscited by 63
  8. 2023Warpformer: A Multi-scale Modeling Approach for Irregular Clinical Time Seriescited by 54
  9. 2020A Survey on Principles, Models and Methods for Learning from Irregularly Sampled Time Seriescited by 42
  10. 2021Dynamic Gaussian Mixture based Deep Generative Model For Robust Forecasting on Sparse Multivariate Time Seriescited by 30
  11. 2022EXIT: Extrapolation and Interpolation-based Neural Controlled Differential Equations for Time-series Classification and Forecastingcited by 22
  12. 2023Irregular Traffic Time Series Forecasting Based on Asynchronous Spatio-Temporal Graph Convolutional Networkcited by 19
  13. 2021ACE-NODE: Attentive Co-Evolving Neural Ordinary Differential Equationscited by 15
  14. 2023Learnable Path in Neural Controlled Differential Equationscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 5.

10 papers.

  1. 2025How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlookcited by 20
  2. 2025Revitalizing Canonical Pre-Alignment for Irregular Multivariate Time Series Forecastingcited by 5
  3. 2025Physics-Informed Teleconnection-Aware Transformer for Global Subseasonal-to-Seasonal Forecastingcited by 2
  4. 2026MILM: Large Language Models for Multimodal Irregular Time Series with Informative Samplingcited by 2
  5. 2025UniExtreme: A Universal Foundation Model for Extreme Weather Forecastingcited by 1
  6. 2026MM-ISTS: Cooperating Irregularly Sampled Time Series Forecasting with Multimodal Vision-Text LLMscited by 1
  7. 2025Unlocking the Potential of Linear Networks for Irregular Multivariate Time Series Forecastingcited by 0
  8. 2026GeoDecider: An Evidence-Grounded Agent for Geological Interpretation via Deliberative Reasoningcited by 0
  9. 2026TRIAGE: Dialectical Reasoning for Explainable Risk Prediction on Irregularly Sampled Medical Time Series with LLMscited by 0
  10. 2026LLMODE: Aligning ODEs with LLMs via Gated Token Injection for Irregular Spatio-Temporal Forecastingcited by 0