mlsanityData to 14 Oct 2026

How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlook

arXiv 2503.11835 ↗First posted 14 Mar 2025cs.LG cs.CVcited by 20

Not on the map. Why?

Abstract (arXiv)

Time series analysis (TSA) is a longstanding research topic in the data mining community and has wide real-world significance. Compared to "richer" modalities such as language and vision, which have recently experienced explosive development and are densely connected, the time-series modality remains relatively underexplored and isolated. We notice that many recent TSA works have formed a new research field, i.e., Multiple Modalities for TSA (MM4TSA). In general, these MM4TSA works follow a common motivation: how TSA can benefit from multiple modalities. This survey is the first to offer a comprehensive review and a detailed outlook for this emerging field. Specifically, we systematically discuss three benefits: (1) reusing foundation models of other modalities for efficient TSA, (2) multimodal extension for enhanced TSA, and (3) cross-modality interaction for advanced TSA. We further group the works by the introduced modality type, including text, images, audio, tables, and others, within each perspective. Finally, we identify the gaps with future opportunities, including the reused modalities selections, heterogeneous modality combinations, and unseen tasks generalizations, corresponding to the three benefits. We release an up-to-date GitHub repository that includes key papers and resources.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 99 of its 217 references that are in the map. First posted on arXiv 14 Mar 2025.

Builds on

99 of 217 references.

References · 99

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  9. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  10. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  11. 2024Chronos: Learning the Language of Time Seriescited by 759
  12. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
38 more
  1. 2021AST: Audio Spectrogram Transformercited by 643
  2. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  3. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 471
  4. 2019Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019cited by 273
  5. 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 212
  6. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 169
  7. 2018Deep Multi-View Spatial-Temporal Network for Taxi Demand Predictioncited by 162
  8. 2024A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalistcited by 137
  9. 2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasterscited by 129
  10. 2023Large Language Models are Few-Shot Health Learnerscited by 118
  11. 2020A Review on Deep Learning Techniques for Video Predictioncited by 114
  12. 2024Large Language Models for Time Series: A Surveycited by 114
  13. 2023ForecastPFN: Synthetically-Trained Zero-Shot Forecastingcited by 104
  14. 2024ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoningcited by 103
  15. 2024ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Datacited by 100
  16. 2024From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflectioncited by 98
  17. 2023Where Would I Go Next? Large Language Models as Human Mobility Predictorscited by 93
  18. 2021Voice2Series: Reprogramming Acoustic Models for Time Series Classificationcited by 91
  19. 2024Language Models Still Struggle to Zero-shot Reason about Time Seriescited by 88
  20. 2024UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Predictioncited by 87
  21. 2024Context is Key: A Benchmark for Forecasting with Essential Textual Informationcited by 87
  22. 2015Imaging Time-Series to Improve Classification and Imputationcited by 85
  23. 2024Spatial-Temporal Large Language Model for Traffic Predictioncited by 83
  24. 2025Time-MQA: Time Series Multi-Task Question Answering with Context Enhancementcited by 82
  25. 2023Time Series as Images: Vision Transformer for Irregularly Sampled Time Seriescited by 79
  26. 2023UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Webcited by 78
  27. 2020Tabular Transformers for Modeling Multivariate Time Seriescited by 77
  28. 2024AutoTimes: Autoregressive Time Series Forecasters via Large Language Modelscited by 65
  29. 2025Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecastingcited by 63
  30. 2023The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challengescited by 59
  31. 2024LSTPrompt: Large Language Models as Zero-Shot Time Series Forecasters by Long-Short-Term Promptingcited by 59
  32. 2022Leveraging Language Foundation Models for Human Mobility Forecastingcited by 50
  33. 2023Multimodal LLMs for health grounded in individual-specific datacited by 49
  34. 2024TimeSeriesExam: A time series understanding examcited by 49
  35. 2018A neural attention model for speech command recognitioncited by 45
  36. 2023TrafficGPT: Viewing, Processing and Interacting with Traffic Foundation Modelscited by 45
  37. 2024Can LLMs Understand Time Series Anomalies?cited by 41
  38. 2025Context-Alignment: Activating and Enhancing LLM Capabilities in Time Seriescited by 39

50 most cited of 99 All 99 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 13; 2026: 7.

20 papers.

Sort:

By area:

  1. 2025TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learningcited by 18
  2. 2025VisionTS++: Cross-Modal Time Series Foundation Model with Continual Pre-trained Vision Backbonescited by 12
  3. 2025AlphaCast: A Human Wisdom-LLM Intelligence Co-Reasoning Framework for Interactive Time Series Forecastingcited by 12
  4. 2026TSAQA: Time Series Analysis Question And Answering Benchmarkcited by 11
  5. 2025CaTS-Bench: Can Language Models Describe Time Series?cited by 10
  6. 2025Time-RA: Towards Time Series Reasoning for Anomaly Diagnosis with LLM Feedbackcited by 8
  7. 2025Who's Your Judge? On the Detectability of LLM-Generated Judgmentscited by 8
  8. 2025From Images to Signals: Are Large Vision Models Useful for Time Series Analysis?cited by 7
  9. 2025TimeRecipe: A Time-Series Forecasting Recipe via Benchmarking Module Level Effectivenesscited by 7
  10. 2026Spectral Text Fusion: A Frequency-Aware Approach to Multimodal Time-Series Forecastingcited by 6
  11. 2026SenTSR-Bench: Thinking with Injected Knowledge for Time-Series Reasoningcited by 2
  12. 2026Rethinking Post-Training Recipes for Multimodal Time-Series Forecastingcited by 2
8 more
  1. 2025ScreenLLM: Stateful Screen Schema for Efficient Action Understanding and Predictioncited by 1
  2. 2025Causal Graph Fuzzy LLMs: A First Introduction and Applications in Time Series Forecastingcited by 1
  3. 2025PA-RNet: Perturbation-Aware Residual Network for Robust Multimodal Time Series Forecastingcited by 1
  4. 2026Multimodal Collaborative Debate for Zero-Shot Time Series Reasoningcited by 1
  5. 2026A Cost-Effective Multimodal LLM Reasoning Framework for Question Answering over Irregular Clinical Time Seriescited by 1
  6. 2025Transforming Decoder-Only Transformers for Accurate WiFi-Telemetry Based Indoor Localizationcited by 0
  7. 2025Indoor Localization using Compact, Telemetry-Agnostic, Transfer-Learning Enabled Decoder-Only Transformercited by 0
  8. 2026PAMF: Prior-Aware Multimodal Fusion for Incomplete Time Series Datacited by 0