mlsanityData to 7 Oct 2026

Hierarchical Multimodal LLMs with Semantic Space Alignment for Enhanced Time Series Classification

arXiv 2410.18686 ↗First posted 24 Oct 2024cs.LGcited by 7

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

Time series classification plays a fundamental role in a wide range of real-world applications. Recently, large language models (LLMs) have demonstrated strong generalization and reasoning capacities, but directly applying them to time series classification remains non-trivial due to the representation gap between numerical sequences and linguistic semantics. In this paper, we propose HiTime, a hierarchical LLM-based framework for multimodal time series classification that bridges structured temporal representations with semantic reasoning in a generative paradigm. Specifically, we design a hierarchical sequence feature encoding module composed of a data-specific encoder and a task-specific encoder to extract complementary temporal features. To mitigate the embedding gap between time series representations and textual semantics, we further introduce a semantic space alignment module that jointly performs coarse-grained global modeling and fine-grained cross-modal correspondence. Building upon the above representations, we employ a parameter-efficient supervised fine-tuning strategy to activate the generative classification capability of the algined LLMs, thereby transforming conventional discriminative time series classification into a generative task. Extensive experiments on multiple benchmarks demonstrate that the proposed framework consistently outperforms state-of-the-art baselines. The code is publicly available at https://github.com/Xiaoyu-Tao/HiTime.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 79 references that are in the map. First posted on arXiv 24 Oct 2024.

Builds on

36 of 79 references.

References · 36

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  3. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  4. 2020Reformer: The Efficient Transformercited by 1,658
  5. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  6. 2024Chronos: Learning the Language of Time Seriescited by 712
  7. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  8. 2018Deep learning for time series classification: a reviewcited by 575
  9. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
  10. 2019InceptionTime: Finding AlexNet for Time Series Classificationcited by 386
  11. 2018The UEA multivariate time series classification archive, 2018cited by 330
  12. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
24 more
  1. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 267
  2. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
  3. 2020MINIROCKET: A Very Fast (Almost) Deterministic Transform for Time Series Classificationcited by 186
  4. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 165
  5. 2020Benchmarking Multivariate Time Series Classification Algorithmscited by 145
  6. 2023Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Surveycited by 140
  7. 2023UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecastingcited by 139
  8. 2016Multi-Scale Convolutional Neural Networks for Time Series Classificationcited by 99
  9. 2013A Time Series Forest for Classification and Feature Extractioncited by 69
  10. 2023Deep Learning for Time Series Classification and Extrinsic Regression: A Current Surveycited by 68
  11. 2024Empowering Time Series Analysis with Large Language Models: A Surveycited by 60
  12. 2024aeon: a Python toolkit for learning from time seriescited by 46
  13. 2018A review on distance based time series classificationcited by 40
  14. 2014Highly comparative feature-based time-series classificationcited by 39
  15. 2017Learning Compact Recurrent Neural Networks with Block-Term Tensor Decompositioncited by 33
  16. 2019Semi-Supervised Hierarchical Recurrent Graph Neural Network for City-Wide Parking Availability Predictioncited by 30
  17. 2024Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Modelscited by 23
  18. 2023FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classificationcited by 21
  19. 2024Exploring Large-Scale Language Models to Evaluate EEG-Based Multimodal Data for Mental Healthcited by 13
  20. 2024Cross-Domain Pre-training with Language Models for Transferable Time Series Representationscited by 12
  21. 2024Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Valuescited by 10
  22. 2024Advancing Time Series Classification with Multimodal Language Modelingcited by 10
  23. 2023ShapeWordNet: An Interpretable Shapelet Neural Network for Physiological Signal Classificationcited by 7
  24. 2017A Shapelet Transform for Multivariate Time Series Classificationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026InstructTime++: Time Series Classification with Multimodal Language Modeling via Implicit Feature Enhancement
  2. 2024TableTime: Reformulating Time Series Classification as Training-Free Table Understanding with Large Language Models
  3. 2024Towards Time Series Reasoning with LLMs
  4. 2025TimeSense:Making Large Language Models Proficient in Time-Series Analysis
  5. 2024ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning
  6. 2025Adapting LLMs to Time Series Forecasting via Temporal Heterogeneity Modeling and Representation Alignment
  7. 2024NeST: Neighborhood-aware semantic alignment and temporal modulation for LLM based time series forecasting
  8. 2025Adapting Large Language Models for Time Series Modeling via a Novel Parameter-efficient Adaptation Method