# Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysis

arXiv: https://arxiv.org/abs/2406.08627 · HTML: https://mlsanity.com/paper/2406.08627/

## Key facts

- Cited by 74 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 83 references that are in the map.
- First posted on arXiv 12 Jun 2024.

## Builds on (most cited first, 30 of 52)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [FLAVA: A Foundational Language And Vision Alignment Model](https://mlsanity.com/paper/2112.04482/) (2021)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Learning Universal Policies via Text-Guided Video Generation](https://mlsanity.com/paper/2302.00111/) (2023)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy](https://mlsanity.com/paper/2110.02642/) (2021)
- [Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting](https://mlsanity.com/paper/2205.14415/) (2022)
- [TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting](https://mlsanity.com/paper/2405.14616/) (2024)
- [Long-term Forecasting with TiDE: Time-series Dense Encoder](https://mlsanity.com/paper/2304.08424/) (2023)
- [Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019](https://mlsanity.com/paper/1911.13288/) (2019)
- [TSMixer: An All-MLP Architecture for Time Series Forecasting](https://mlsanity.com/paper/2303.06053/) (2023)

## Built on by (most cited first, 30 of 74)

- [MTBench: A Multimodal Time Series Benchmark for Temporal Reasoning and Question Answering](https://mlsanity.com/paper/2503.16858/) (2025)
- [Toto 2.0: Time Series Forecasting Enters the Scaling Era](https://mlsanity.com/paper/2605.20119/) (2026)
- [Aurora: Towards Universal Generative Multimodal Time Series Forecasting](https://mlsanity.com/paper/2509.22295/) (2025)
- [A Survey of Reasoning and Agentic Systems in Time Series with Large Language Models](https://mlsanity.com/paper/2509.11575/) (2025)
- [BRIDGE: Bootstrapping Text to Control Time-Series Generation via Multi-Agent Iterative Optimization and Diffusion Modeling](https://mlsanity.com/paper/2503.02445/) (2025)
- [Time-IMM: A Dataset and Benchmark for Irregular Multimodal Multivariate Time Series](https://mlsanity.com/paper/2506.10412/) (2025)
- [SciTS: Scientific Time Series Understanding and Generation with LLMs](https://mlsanity.com/paper/2510.03255/) (2025)
- [When LLM Meets Time Series: Can LLMs Perform Multi-Step Time Series Reasoning and Inference](https://mlsanity.com/paper/2509.01822/) (2025)
- [Fidel-TS: A High-Fidelity Multimodal Benchmark for Time Series Forecasting](https://mlsanity.com/paper/2509.24789/) (2025)
- [Multimodal Conditioned Diffusive Time Series Forecasting](https://mlsanity.com/paper/2504.19669/) (2025)
- [Position: Beyond Model-Centric Prediction -- Agentic Time Series Forecasting](https://mlsanity.com/paper/2602.01776/) (2026)
- [MoTime: A Dataset Suite for Multimodal Time Series Forecasting](https://mlsanity.com/paper/2505.15072/) (2025)
- [TimeRecipe: A Time-Series Forecasting Recipe via Benchmarking Module Level Effectiveness](https://mlsanity.com/paper/2506.06482/) (2025)
- [TRACE: Grounding Time Series in Context for Multimodal Embedding and Retrieval](https://mlsanity.com/paper/2506.09114/) (2025)
- [UniCA: Unified Covariate Adaptation for Time Series Foundation Model](https://mlsanity.com/paper/2506.22039/) (2025)
- [BALM-TSF: Balanced Multimodal Alignment for LLM-Based Time Series Forecasting](https://mlsanity.com/paper/2509.00622/) (2025)
- [CoRA: Covariate-Aware Adaptation of Time Series Foundation Models](https://mlsanity.com/paper/2510.12681/) (2025)
- [Spectral Text Fusion: A Frequency-Aware Approach to Multimodal Time-Series Forecasting](https://mlsanity.com/paper/2602.01588/) (2026)
- [Eliciting Chain-of-Thought Reasoning for Time Series Analysis using Reinforcement Learning](https://mlsanity.com/paper/2510.01116/) (2025)
- [TelecomTS: A Multi-Modal Observability Dataset for Time Series and Language Analysis](https://mlsanity.com/paper/2510.06063/) (2025)
- [QuAnTS: Question Answering on Time Series](https://mlsanity.com/paper/2511.05124/) (2025)
- [FinTexTS: Financial Text-Paired Time-Series Dataset via Semantic-Based and Multi-Level Pairing](https://mlsanity.com/paper/2603.02702/) (2026)
- [Overcoming the Modality Gap in Context-Aided Forecasting](https://mlsanity.com/paper/2603.12451/) (2026)
- [From Text to Forecasts: Bridging Modality Gap with Temporal Evolution Semantic Space](https://mlsanity.com/paper/2603.12664/) (2026)
- [Unlocking the Value of Text: Event-Driven Reasoning and Multi-Level Alignment for Time Series Forecasting](https://mlsanity.com/paper/2603.15452/) (2026)
- [Context-Aware Probabilistic Modeling with LLM for Multimodal Time Series Forecasting](https://mlsanity.com/paper/2505.10774/) (2025)
- [DELPHYNE: A Pre-Trained Model for General and Financial Time Series](https://mlsanity.com/paper/2506.06288/) (2025)
- [AXIS: Explainable Time Series Anomaly Detection with Large Language Models](https://mlsanity.com/paper/2509.24378/) (2025)
- [TiMi: Empower Time Series Transformers with Multimodal Mixture of Experts](https://mlsanity.com/paper/2602.21693/) (2026)
- [Language Modeling for the Future of Finance: A Survey into Metrics, Tasks, and Data Opportunities](https://mlsanity.com/paper/2504.07274/) (2025)

## Abstract (arXiv)

Time series data are ubiquitous across a wide range of real-world domains. While real-world time series analysis (TSA) requires human experts to integrate numerical series data with multimodal domain-specific knowledge, most existing TSA models rely solely on numerical data, overlooking the significance of information beyond numerical series. This oversight is due to the untapped potential of textual series data and the absence of a comprehensive, high-quality multimodal dataset. To overcome this obstacle, we introduce Time-MMD, the first multi-domain, multimodal time series dataset covering 9 primary data domains. Time-MMD ensures fine-grained modality alignment, eliminates data contamination, and provides high usability. Additionally, we develop MM-TSFlib, the first-cut multimodal time-series forecasting (TSF) library, seamlessly pipelining multimodal TSF evaluations based on Time-MMD for in-depth analyses. Extensive experiments conducted on Time-MMD through MM-TSFlib demonstrate significant performance enhancements by extending unimodal TSF to multimodality, evidenced by over 15% mean squared error reduction in general, and up to 40% in domains with rich textual data. More importantly, our datasets and library revolutionize broader applications, impacts, research topics to advance TSA. The dataset is available at https://github.com/AdityaLab/Time-MMD.