mlsanityData to 14 Oct 2026

Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysis

arXiv 2406.08627 ↗First posted 12 Jun 2024cs.LG cs.CLcited by 74

Not on the map. Why?

Abstract (arXiv)

Time series data are ubiquitous across a wide range of real-world domains. While real-world time series analysis (TSA) requires human experts to integrate numerical series data with multimodal domain-specific knowledge, most existing TSA models rely solely on numerical data, overlooking the significance of information beyond numerical series. This oversight is due to the untapped potential of textual series data and the absence of a comprehensive, high-quality multimodal dataset. To overcome this obstacle, we introduce Time-MMD, the first multi-domain, multimodal time series dataset covering 9 primary data domains. Time-MMD ensures fine-grained modality alignment, eliminates data contamination, and provides high usability. Additionally, we develop MM-TSFlib, the first-cut multimodal time-series forecasting (TSF) library, seamlessly pipelining multimodal TSF evaluations based on Time-MMD for in-depth analyses. Extensive experiments conducted on Time-MMD through MM-TSFlib demonstrate significant performance enhancements by extending unimodal TSF to multimodality, evidenced by over 15% mean squared error reduction in general, and up to 40% in domains with rich textual data. More importantly, our datasets and library revolutionize broader applications, impacts, research topics to advance TSA. The dataset is available at https://github.com/AdityaLab/Time-MMD.

Key facts. Cited by 74 papers in the map (data through 14 Oct 2026). Builds on 52 of its 83 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

52 of 83 references.

References · 52

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  6. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  7. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  8. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  9. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  10. 2020Reformer: The Efficient Transformercited by 1,668
  11. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  12. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
38 more
  1. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  2. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
  3. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 761
  4. 2024Chronos: Learning the Language of Time Seriescited by 759
  5. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  6. 2023A decoder-only foundation model for time-series forecastingcited by 692
  7. 2014Distributed Representations of Sentences and Documentscited by 678
  8. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 649
  9. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  10. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 554
  11. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  12. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  13. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 394
  14. 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 391
  15. 2024TimeMixer: Decomposable Multiscale Mixing for Time Series Forecastingcited by 346
  16. 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 332
  17. 2019Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019cited by 273
  18. 2023TSMixer: An All-MLP Architecture for Time Series Forecastingcited by 240
  19. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 208
  20. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 174
  21. 2019GP-VAE: Deep Probabilistic Time Series Imputationcited by 172
  22. 2023Frequency-domain MLPs are More Effective Learners in Time Series Forecastingcited by 172
  23. 2022Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structurescited by 160
  24. 2022ETSformer: Exponential Smoothing Transformers for Time-series Forecastingcited by 148
  25. 2023Make-Your-Video: Customized Video Generation Using Textual and Structural Guidancecited by 129
  26. 2023Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictorscited by 113
  27. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 99
  28. 2023Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queriescited by 82
  29. 2023Temporal Data Meets LLM -- Explainable Financial Time Series Forecastingcited by 77
  30. 2023Frozen Language Model Helps ECG Zero-Shot Learningcited by 62
  31. 2024LSTPrompt: Large Language Models as Zero-Shot Time Series Forecasters by Long-Short-Term Promptingcited by 59
  32. 2023SegRNN: Segment Recurrent Neural Network for Long-Term Time Series Forecastingcited by 55
  33. 2021Towards Unsupervised Domain Generalizationcited by 40
  34. 2022Modeling Temporal Data as Continuous Functions with Stochastic Process Diffusioncited by 39
  35. 2021Missing Value Imputation on Multidimensional Time Seriescited by 33
  36. 2023Large Pre-trained time series models for cross-domain Time series analysis taskscited by 23
  37. 2023SEntFiN 1.0: Entity-Aware Sentiment Analysis for Financial Newscited by 18
  38. 2021When in Doubt: Neural Non-Parametric Uncertainty Quantification for Epidemic Forecastingcited by 16

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 29; 2026: 44.

74 papers.

Sort:

By area:

  1. 2025MTBench: A Multimodal Time Series Benchmark for Temporal Reasoning and Question Answeringcited by 55
  2. 2026Toto 2.0: Time Series Forecasting Enters the Scaling Eracited by 30
  3. 2025Aurora: Towards Universal Generative Multimodal Time Series Forecastingcited by 29
  4. 2025A Survey of Reasoning and Agentic Systems in Time Series with Large Language Modelscited by 21
  5. 2025BRIDGE: Bootstrapping Text to Control Time-Series Generation via Multi-Agent Iterative Optimization and Diffusion Modelingcited by 19
  6. 2025Time-IMM: A Dataset and Benchmark for Irregular Multimodal Multivariate Time Seriescited by 19
  7. 2025SciTS: Scientific Time Series Understanding and Generation with LLMscited by 15
  8. 2025When LLM Meets Time Series: Can LLMs Perform Multi-Step Time Series Reasoning and Inferencecited by 12
  9. 2025Fidel-TS: A High-Fidelity Multimodal Benchmark for Time Series Forecastingcited by 10
  10. 2025Multimodal Conditioned Diffusive Time Series Forecastingcited by 9
  11. 2026Position: Beyond Model-Centric Prediction -- Agentic Time Series Forecastingcited by 9
  12. 2025MoTime: A Dataset Suite for Multimodal Time Series Forecastingcited by 7
38 more
  1. 2025TimeRecipe: A Time-Series Forecasting Recipe via Benchmarking Module Level Effectivenesscited by 7
  2. 2025TRACE: Grounding Time Series in Context for Multimodal Embedding and Retrievalcited by 7
  3. 2025UniCA: Unified Covariate Adaptation for Time Series Foundation Modelcited by 7
  4. 2025BALM-TSF: Balanced Multimodal Alignment for LLM-Based Time Series Forecastingcited by 6
  5. 2025CoRA: Covariate-Aware Adaptation of Time Series Foundation Modelscited by 6
  6. 2026Spectral Text Fusion: A Frequency-Aware Approach to Multimodal Time-Series Forecastingcited by 6
  7. 2025Eliciting Chain-of-Thought Reasoning for Time Series Analysis using Reinforcement Learningcited by 5
  8. 2025TelecomTS: A Multi-Modal Observability Dataset for Time Series and Language Analysiscited by 5
  9. 2025QuAnTS: Question Answering on Time Seriescited by 5
  10. 2026FinTexTS: Financial Text-Paired Time-Series Dataset via Semantic-Based and Multi-Level Pairingcited by 5
  11. 2026Overcoming the Modality Gap in Context-Aided Forecastingcited by 5
  12. 2026From Text to Forecasts: Bridging Modality Gap with Temporal Evolution Semantic Spacecited by 5
  13. 2026Unlocking the Value of Text: Event-Driven Reasoning and Multi-Level Alignment for Time Series Forecastingcited by 5
  14. 2025Context-Aware Probabilistic Modeling with LLM for Multimodal Time Series Forecastingcited by 4
  15. 2025DELPHYNE: A Pre-Trained Model for General and Financial Time Seriescited by 4
  16. 2025AXIS: Explainable Time Series Anomaly Detection with Large Language Modelscited by 4
  17. 2026TiMi: Empower Time Series Transformers with Multimodal Mixture of Expertscited by 4
  18. 2025Language Modeling for the Future of Finance: A Survey into Metrics, Tasks, and Data Opportunitiescited by 3
  19. 2025Fusing Narrative Semantics for Financial Volatility Forecastingcited by 3
  20. 2026Empowering Time Series Analysis with Large-Scale Multimodal Pretrainingcited by 3
  21. 2026Time Series Reasoning via Process-Verifiable Thinking Data Synthesis and Scheduling for Tailored LLM Reasoningcited by 3
  22. 2026Does Text Actually Help? Uncovering and Resolving Text Collapse in Multimodal Time Series Forecastingcited by 3
  23. 2026EXAONE Finance 1.0: An Attention-free Time Series Foundation Model for Financial Time Seriescited by 3
  24. 2024LTSM-Bundle: A Toolbox and Benchmark on Large Language Models for Time Series Forecastingcited by 2
  25. 2025Spiking Neural Networks with Temporal Attention-Guided Adaptive Fusion for imbalanced Multi-modal Learningcited by 2
  26. 2025Improving Hospital Risk Prediction with Knowledge-Augmented Multimodal EHR Modelingcited by 2
  27. 2026LEAF: A Living Benchmark for Event-Augmented Forecastingcited by 2
  28. 2026AION: Next-Generation Tasks and Practical Harness for Time Seriescited by 2
  29. 2026Rethinking Post-Training Recipes for Multimodal Time-Series Forecastingcited by 2
  30. 2025ScreenLLM: Stateful Screen Schema for Efficient Action Understanding and Predictioncited by 1
  31. 2025TAT: Temporal-Aligned Transformer for Multi-Horizon Peak Demand Forecastingcited by 1
  32. 2025PA-RNet: Perturbation-Aware Residual Network for Robust Multimodal Time Series Forecastingcited by 1
  33. 2025LLM-Integrated Bayesian State Space Models for Multimodal Time-Series Forecastingcited by 1
  34. 2026Multi-Modal Time Series Prediction via Mixture of Modulated Expertscited by 1
  35. 2026In-context Pre-trained Time-Series Foundation Models adapt to Unseen Taskscited by 1
  36. 2026Spectral-Aware Text-to-Time Series Generation with Billion-Scale Multimodal Meteorological Datacited by 1
  37. 2026LLM as Forecasting Planner: Training-Free Text Conditioning for Time-Series Foundation Modelscited by 1
  38. 2026QiYao-M: Multimodal Time Series Foundation Model with Role-Aware Modeling of Endogenous and Exogenous Modalitiescited by 1

50 most cited of 74 All 74 (JSON)