# ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning

arXiv: https://arxiv.org/abs/2412.03104 · HTML: https://mlsanity.com/paper/2412.03104/

## Key facts

- Cited by 103 papers in the map (data through 14 Oct 2026).
- Placed in Time series reasoning, one of 17 directions in LLM reasoning.
- Builds on 24 of its 60 references that are in the map.
- First posted on arXiv 4 Dec 2024.

## Placed in

- LLM reasoning / [Time series reasoning](https://mlsanity.com/lr/time-series-reasoning/)
- Synthetic data and self-improvement for LLMs / [Model collapse from synthetic data](https://mlsanity.com/synth/model-collapse-from-synthetic-data/)

## Builds on (most cited first, 24 of 24)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://mlsanity.com/paper/2311.07919/) (2023)
- [Time Series Forecasting With Deep Learning: A Survey](https://mlsanity.com/paper/2004.13408/) (2020)
- [AudioPaLM: A Large Language Model That Can Speak and Listen](https://mlsanity.com/paper/2306.12925/) (2023)
- [ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels](https://mlsanity.com/paper/1910.13051/) (2019)
- [Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019](https://mlsanity.com/paper/1911.13288/) (2019)
- [Language Models Still Struggle to Zero-shot Reason about Time Series](https://mlsanity.com/paper/2404.11757/) (2024)
- [VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2408.17253/) (2024)
- [GraphTranslator: Aligning Graph Model to Large Language Model for Open-ended Tasks](https://mlsanity.com/paper/2402.07197/) (2024)
- [ECG-QA: A Comprehensive Question Answering Dataset Combined With Electrocardiogram](https://mlsanity.com/paper/2306.15681/) (2023)
- [Time Series Contrastive Learning with Information-Aware Augmentations](https://mlsanity.com/paper/2303.11911/) (2023)
- [Open-TI: Open Traffic Intelligence with Augmented Language Model](https://mlsanity.com/paper/2401.00211/) (2023)
- [Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmark](https://mlsanity.com/paper/2404.16563/) (2024)
- [Towards Time Series Reasoning with LLMs](https://mlsanity.com/paper/2409.11376/) (2024)
- [Large language models can be zero-shot anomaly detectors for time series?](https://mlsanity.com/paper/2405.14755/) (2024)
- [Constructing Large-Scale Real-World Benchmark Datasets for AIOps](https://mlsanity.com/paper/2208.03938/) (2022)
- [See it, Think it, Sorted: Large Multimodal Models are Few-shot Time Series Anomaly Analyzers](https://mlsanity.com/paper/2411.02465/) (2024)
- [OneShotSTL: One-Shot Seasonal-Trend Decomposition For Online Time Series Anomaly Detection And Forecasting](https://mlsanity.com/paper/2304.01506/) (2023)
- [Text2TimeSeries: Enhancing Financial Forecasting through Time Series Prediction Updates with Event-Driven Insights from Large Language Models](https://mlsanity.com/paper/2407.03689/) (2024)
- [Are Synthetic Time-series Data Really not as Good as Real Data?](https://mlsanity.com/paper/2402.00607/) (2024)
- [A Survey of Time Series Anomaly Detection Methods in the AIOps Domain](https://mlsanity.com/paper/2308.00393/) (2023)

## Built on by (most cited first, 30 of 103)

- [Time-MQA: Time Series Multi-Task Question Answering with Context Enhancement](https://mlsanity.com/paper/2503.01875/) (2025)
- [Moirai 2.0: When Less Is More for Time Series Forecasting](https://mlsanity.com/paper/2511.11698/) (2025)
- [Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrative](https://mlsanity.com/paper/2502.08942/) (2025)
- [TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop](https://mlsanity.com/paper/2503.01013/) (2025)
- [Aurora: Towards Universal Generative Multimodal Time Series Forecasting](https://mlsanity.com/paper/2509.22295/) (2025)
- [TimeART: Towards Agentic Time Series Reasoning via Tool-Augmentation](https://mlsanity.com/paper/2601.13653/) (2026)
- [A Survey of Reasoning and Agentic Systems in Time Series with Large Language Models](https://mlsanity.com/paper/2509.11575/) (2025)
- [How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlook](https://mlsanity.com/paper/2503.11835/) (2025)
- [Time-IMM: A Dataset and Benchmark for Irregular Multimodal Multivariate Time Series](https://mlsanity.com/paper/2506.10412/) (2025)
- [TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learning](https://mlsanity.com/paper/2506.13705/) (2025)
- [When Language Overrules: Revealing Text Dominance in Multimodal Large Language Models](https://mlsanity.com/paper/2508.10552/) (2025)
- [TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Models](https://mlsanity.com/paper/2601.18744/) (2026)
- [LLM/Agent-as-Data-Analyst: A Survey](https://mlsanity.com/paper/2509.23988/) (2025)
- [Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Models](https://mlsanity.com/paper/2503.11411/) (2025)
- [SciTS: Scientific Time Series Understanding and Generation with LLMs](https://mlsanity.com/paper/2510.03255/) (2025)
- [BEDTime: A Unified Benchmark for Automatically Describing Time Series](https://mlsanity.com/paper/2509.05215/) (2025)
- [TS-Reasoner: Aligning Time Series Foundation Models with LLM Reasoning](https://mlsanity.com/paper/2510.03519/) (2025)
- [TSAQA: Time Series Analysis Question And Answering Benchmark](https://mlsanity.com/paper/2601.23204/) (2026)
- [Inferring Events from Time Series using Language Models](https://mlsanity.com/paper/2503.14190/) (2025)
- [Towards Foundation Models for Zero-Shot Time Series Anomaly Detection: Leveraging Synthetic Data and Relative Context Discrepancy](https://mlsanity.com/paper/2509.21190/) (2025)
- [TS-Agent: Understanding and Reasoning Over Raw Time Series via Iterative Insight Gathering](https://mlsanity.com/paper/2510.07432/) (2025)
- [A Survey on Efficient Large Language Model Training: From Data-centric Perspectives](https://mlsanity.com/paper/2510.25817/) (2025)
- [Enhancing LLM Reasoning for Time Series Classification by Tailored Thinking and Fused Decision](https://mlsanity.com/paper/2506.00807/) (2025)
- [TempoGPT: Enhancing Time Series Reasoning via Quantizing Embedding](https://mlsanity.com/paper/2501.07335/) (2025)
- [Chat-TS: Enhancing Multi-Modal Reasoning Over Time-Series and Natural Language Data](https://mlsanity.com/paper/2503.10883/) (2025)
- [TRACE: Grounding Time Series in Context for Multimodal Embedding and Retrieval](https://mlsanity.com/paper/2506.09114/) (2025)
- [STReasoner: Empowering LLMs for Spatio-Temporal Reasoning in Time Series via Spatial-Aware Reinforcement Learning](https://mlsanity.com/paper/2601.03248/) (2026)
- [TimeOmni-VL: Unified Models for Time Series Understanding and Generation](https://mlsanity.com/paper/2602.17149/) (2026)
- [Can Large Language Models Adequately Perform Symbolic Reasoning Over Time Series?](https://mlsanity.com/paper/2508.03963/) (2025)
- [ARFBench: Benchmarking Time Series Question Answering Ability for Software Incident Response](https://mlsanity.com/paper/2604.21199/) (2026)

## Abstract (arXiv)

Understanding time series is crucial for its application in real-world scenarios. Recently, large language models (LLMs) have been increasingly applied to time series tasks, leveraging their strong language capabilities to enhance various applications. However, research on multimodal LLMs (MLLMs) for time series understanding and reasoning remains limited, primarily due to the scarcity of high-quality datasets that align time series with textual information. This paper introduces ChatTS, a novel MLLM designed for time series analysis. ChatTS treats time series as a modality, similar to how vision MLLMs process images, enabling it to perform both understanding and reasoning with time series. To address the scarcity of training data, we propose an attribute-based method for generating synthetic time series with detailed attribute descriptions. We further introduce Time Series Evol-Instruct, a novel approach that generates diverse time series Q&As, enhancing the model's reasoning capabilities. To the best of our knowledge, ChatTS is the first TS-MLLM that takes multivariate time series as input for understanding and reasoning, which is fine-tuned exclusively on synthetic datasets. We evaluate its performance using benchmark datasets with real-world data, including six alignment tasks and four reasoning tasks. Our results show that ChatTS significantly outperforms existing vision-based MLLMs (e.g., GPT-4o) and text/agent-based LLMs, achieving a 46.0% improvement in alignment tasks and a 25.8% improvement in reasoning tasks. We have open-sourced the source code, model checkpoint and datasets at https://github.com/NetManAIOps/ChatTS.