mlsanityData to 14 Oct 2026

Towards Interpretable Time Series Foundation Models

arXiv 2507.07439 ↗First posted 10 Jul 2025cs.CL cs.AIcited by 3

On the map: LLM reasoning → Time series reasoning

Abstract (arXiv)

In this paper, we investigate the distillation of time series reasoning capabilities into small, instruction-tuned language models as a step toward building interpretable time series foundation models. Leveraging a synthetic dataset of mean-reverting time series with systematically varied trends and noise levels, we generate natural language annotations using a large multimodal model and use these to supervise the fine-tuning of compact Qwen models. We introduce evaluation metrics that assess the quality of the distilled reasoning - focusing on trend direction, noise intensity, and extremum localization - and show that the post-trained models acquire meaningful interpretive capabilities. Our results highlight the feasibility of compressing time series understanding into lightweight, language-capable models suitable for on-device or privacy-sensitive deployment. This work contributes a concrete foundation toward developing small, interpretable models that explain temporal patterns in natural language.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Time series reasoning, one of 17 directions in LLM reasoning. Builds on 15 of its 19 references that are in the map. First posted on arXiv 10 Jul 2025.

Builds on

15 of 19 references.

References · 15

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  3. 2024Chronos: Learning the Language of Time Seriescited by 759
  4. 2023A decoder-only foundation model for time-series forecastingcited by 692
  5. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  6. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
  7. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  8. 2023How well do Large Language Models perform in Arithmetic tasks?cited by 130
  9. 2024UniTS: A Unified Multi-Task Time Series Modelcited by 94
  10. 2024Position: What Can Large Language Models Tell Us about Time Series Analysiscited by 47
  11. 2024Can LLMs Understand Time Series Anomalies?cited by 41
  12. 2024Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmarkcited by 38
3 more
  1. 2023The first step is the hardest: Pitfalls of Representing and Tokenizing Temporal Data for Large Language Modelscited by 34
  2. 2019Explaining Agent-Based Financial Market Simulationcited by 10
  3. 2023Multi-Modal Financial Time-Series Retrieval Through Latent Space Projectionscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Explainable Load Forecasting with Covariate-Informed Time Series Foundation Modelscited by 4
  2. 2026Dissecting Chronos: Sparse Autoencoders Reveal Causal Feature Hierarchies in Time Series Foundation Modelscited by 2
  3. 2026Discovering and Preserving Category Correlation Knowledge via Adaptive Reciprocal Knowledge Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Achieving Time Series Reasoning Requires Rethinking Model Design, Tasks Formulation, and Evaluation
  2. 2025The language of time: a language model perspective on time-series foundation models
  3. 2026Thoth: Mid-Training Bridges LLMs to Time Series Understanding
  4. 2025TimeSense:Making Large Language Models Proficient in Time-Series Analysis
  5. 2026TimeSeriesExamAgent: Creating Time Series Reasoning Benchmarks at Scale
  6. 2025Conversational Time Series Foundation Models: Towards Explainable and Effective Forecasting
  7. 2024Towards Time Series Reasoning with LLMs
  8. 2026Lapras: Latent Reasoning for Time Series Language Models