mlsanityData to 14 Oct 2026

Time-IMM: A Dataset and Benchmark for Irregular Multimodal Multivariate Time Series

arXiv 2506.10412 ↗First posted 12 Jun 2025cs.LG cs.AI cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

Time series data in real-world applications such as healthcare, climate modeling, and finance are often irregular, multimodal, and messy, with varying sampling rates, asynchronous modalities, and pervasive missingness. However, existing benchmarks typically assume clean, regularly sampled, unimodal data, creating a significant gap between research and real-world deployment. We introduce Time-IMM, a dataset specifically designed to capture cause-driven irregularity in multimodal multivariate time series. Time-IMM represents nine distinct types of time series irregularity, categorized into trigger-based, constraint-based, and artifact-based mechanisms. Complementing the dataset, we introduce IMM-TSF, a benchmark library for forecasting on irregular multimodal time series, enabling asynchronous integration and realistic evaluation. IMM-TSF includes specialized fusion modules, including a timestamp-to-text fusion module and a multimodality fusion module, which support both recency-aware averaging and attention-based integration strategies. Empirical results demonstrate that explicitly modeling multimodality on irregular time series data leads to substantial gains in forecasting performance. Time-IMM and IMM-TSF provide a foundation for advancing time series analysis under real-world conditions. The dataset is publicly available at https://github.com/blacksnail789521/Time-IMM, and the benchmark library can be accessed at https://github.com/blacksnail789521/IMM-TSF. Project page: https://blacksnail789521.github.io/time-imm-project-page/

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 45 of its 85 references that are in the map. First posted on arXiv 12 Jun 2025.

Builds on

45 of 85 references.

References · 45

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  4. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  5. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  6. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  7. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  8. 2014Distributed Representations of Sentences and Documentscited by 678
  9. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
  10. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 471
  11. 2020Time Series Forecasting With Deep Learning: A Surveycited by 459
  12. 2024TimeMixer: Decomposable Multiscale Mixing for Time Series Forecastingcited by 346
33 more
  1. 2018LRS3-TED: a large-scale dataset for visual speech recognitioncited by 308
  2. 2018Scalable and accurate deep learning for electronic health recordscited by 280
  3. 2024TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methodscited by 249
  4. 2021Multi-Time Attention Networks for Irregularly Sampled Time Seriescited by 238
  5. 2019GRU-ODE-Bayes: Continuous modeling of sporadically-observed time seriescited by 231
  6. 2019Time2Vec: Learning a Vector Representation of Timecited by 213
  7. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 197
  8. 2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasterscited by 129
  9. 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 127
  10. 2024CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuningcited by 108
  11. 2021Modeling Irregular Time Series with Continuous Recurrent Unitscited by 103
  12. 2024ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoningcited by 103
  13. 2021Neural Flows: Efficient Alternative to Neural ODEscited by 101
  14. 2024ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Datacited by 100
  15. 2024From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflectioncited by 98
  16. 2024Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Taskscited by 84
  17. 2016Modeling Missing Data in Clinical Time Series with RNNscited by 80
  18. 2024Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysiscited by 74
  19. 2024Deep Learning for Multivariate Time Series Imputation: A Surveycited by 57
  20. 2025MTBench: A Multimodal Time Series Benchmark for Temporal Reasoning and Question Answeringcited by 55
  21. 2022MedFuse: Multi-modal fusion with clinical time-series data and chest X-ray imagescited by 54
  22. 2025TS-RAG: Retrieval-Augmented Generation based Time Series Foundation Models are Stronger Zero-Shot Forecastercited by 40
  23. 2024Towards Time Series Reasoning with LLMscited by 37
  24. 2022Improving Medical Predictions by Irregular Multimodal Electronic Health Records Modelingcited by 33
  25. 2024Root Cause Analysis In Microservice Using Neural Granger Causal Discoverycited by 32
  26. 2023Deep Imputation of Missing Values in Time Series Health Data: A Review with Benchmarkingcited by 30
  27. 2025Multi-modal Time Series Analysis: A Tutorial and Surveycited by 24
  28. 2024TimeSeriesBench: An Industrial-Grade Benchmark for Time Series Anomaly Detection Modelscited by 23
  29. 2024Multi-Modal Forecaster: Jointly Predicting Time Series and Textual Datacited by 22
  30. 2024TimeRAG: BOOSTING LLM Time Series Forecasting via Retrieval-Augmented Generationcited by 21
  31. 2021MugRep: A Multi-Task Hierarchical Graph Representation Learning Framework for Real Estate Appraisalcited by 20
  32. 2024CESNET-TimeSeries24: Time Series Dataset for Network Traffic Anomaly Detection and Forecastingcited by 12
  33. 2024COKE: Causal Discovery with Chronological Order and Expert Knowledge in High Proportion of Missing Manufacturing Datacited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 16.

19 papers.

Sort:

By area:

  1. 2026Toto 2.0: Time Series Forecasting Enters the Scaling Eracited by 30
  2. 2025TimeSeriesScientist: A General-Purpose AI Agent for Time Series Analysiscited by 22
  3. 2025A Survey of Reasoning and Agentic Systems in Time Series with Large Language Modelscited by 21
  4. 2026TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Modelscited by 18
  5. 2026From Text to Forecasts: Bridging Modality Gap with Temporal Evolution Semantic Spacecited by 5
  6. 2026TiMi: Empower Time Series Transformers with Multimodal Mixture of Expertscited by 4
  7. 2026MILM: Large Language Models for Multimodal Irregular Time Series with Informative Samplingcited by 2
  8. 2026Multi-Modal Time Series Prediction via Mixture of Modulated Expertscited by 1
  9. 2026Spectral-Aware Text-to-Time Series Generation with Billion-Scale Multimodal Meteorological Datacited by 1
  10. 2026Towards Verifiable Agentic Data Science: Solving Irregular TSQA Via Tool-Grounded Reasoningcited by 1
  11. 2025Perseus: Interactive Time Series Segmentation with Sparse Supervision via Stateful Memorycited by 0
  12. 2026ARTEMIS: A Neuro Symbolic Framework for Economically Constrained Market Dynamicscited by 0
7 more
  1. 2026SURGE: An Event-Centric Social Media Sentiment Time Series Benchmark with Interaction Structurecited by 0
  2. 2026QuITE: Query-Based Irregular Time Series Embeddingcited by 0
  3. 2026A Hybrid Attention Model Learning Unified Time-aware Patch Representation for Irregular Multivariate Time Series Forecastingcited by 0
  4. 2026BITS: Rethinking Fair and Comprehensive Evaluation for Irregular Time Series Forecastingcited by 0
  5. 2026TRACE: A Reproducible Benchmark for Electricity Price Forecasting with Official Operational Textcited by 0
  6. 2026CESNET-TimeSeries26: A Multi-Year Network Traffic Time Series Dataset for Annual Seasonality, Concept Drift, and Long-Term Anomaly Detectioncited by 0
  7. 2026MMTA: Benchmarking Multimodal Temporal Analysis with Time Series, Text, and Visioncited by 0