# From Values to Tokens: An LLM-Driven Framework for Context-aware Time Series Forecasting via Symbolic Discretization

arXiv: https://arxiv.org/abs/2508.09191 · HTML: https://mlsanity.com/paper/2508.09191/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 52 references that are in the map.
- First posted on arXiv 8 Aug 2025.

## Builds on (most cited first, 29 of 29)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting](https://mlsanity.com/paper/2405.14616/) (2024)
- [Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts](https://mlsanity.com/paper/2409.16040/) (2024)
- [TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods](https://mlsanity.com/paper/2403.20150/) (2024)
- [TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting](https://mlsanity.com/paper/2310.04948/) (2023)
- [TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series](https://mlsanity.com/paper/2308.08241/) (2023)
- [LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters](https://mlsanity.com/paper/2308.08469/) (2023)
- [SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling](https://mlsanity.com/paper/2302.00861/) (2023)
- [CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning](https://mlsanity.com/paper/2403.07300/) (2024)
- [Temporal Relational Ranking for Stock Prediction](https://mlsanity.com/paper/1809.09441/) (2018)
- [Context is Key: A Benchmark for Forecasting with Essential Textual Information](https://mlsanity.com/paper/2410.18959/) (2024)
- [Deep Factors for Forecasting](https://mlsanity.com/paper/1905.12417/) (2019)
- [LSTPrompt: Large Language Models as Zero-Shot Time Series Forecasters by Long-Short-Term Prompting](https://mlsanity.com/paper/2402.16132/) (2024)
- [Context-Alignment: Activating and Enhancing LLM Capabilities in Time Series](https://mlsanity.com/paper/2501.03747/) (2025)
- [ConvTimeNet: A Deep Hierarchical Fully Convolutional Model for Multivariate Time Series Analysis](https://mlsanity.com/paper/2403.01493/) (2024)
- [Multi-modal Time Series Analysis: A Tutorial and Survey](https://mlsanity.com/paper/2503.13709/) (2025)
- [TimeDP: Learning to Generate Multi-Domain Time Series with Domain Prompts](https://mlsanity.com/paper/2501.05403/) (2025)
- [TimeDART: A Diffusion Autoregressive Transformer for Self-Supervised Time Series Representation](https://mlsanity.com/paper/2410.05711/) (2024)
- [Cross-Domain Pre-training with Language Models for Transferable Time Series Representations](https://mlsanity.com/paper/2403.12372/) (2024)
- [Advancing Time Series Classification with Multimodal Language Modeling](https://mlsanity.com/paper/2403.12371/) (2024)

## Built on by (most cited first, 13 of 13)

- [Cast-R1: Learning Tool-Augmented Sequential Decision Policies for Time Series Forecasting](https://mlsanity.com/paper/2602.13802/) (2026)
- [Position: Beyond Model-Centric Prediction -- Agentic Time Series Forecasting](https://mlsanity.com/paper/2602.01776/) (2026)
- [MemCast: Memory-Driven Time Series Forecasting with Experience-Conditioned Reasoning](https://mlsanity.com/paper/2602.03164/) (2026)
- [Rethinking the Role of LLMs in Time Series Forecasting](https://mlsanity.com/paper/2602.14744/) (2026)
- [Reasoning-Aware Training for Time Series Forecasting](https://mlsanity.com/paper/2605.08625/) (2026)
- [TimeFlow: Towards Stochastic-Aware and Efficient Time Series Generation via Flow Matching Modeling](https://mlsanity.com/paper/2511.07968/) (2025)
- [InstructTime++: Time Series Classification with Multimodal Language Modeling via Implicit Feature Enhancement](https://mlsanity.com/paper/2601.14968/) (2026)
- [CastFlow: Learning Role-Specialized Agentic Workflows for Time Series Forecasting](https://mlsanity.com/paper/2604.27840/) (2026)
- [TableMind++: An Uncertainty-Aware Programmatic Agent for Tool-Augmented Table Reasoning](https://mlsanity.com/paper/2603.07528/) (2026)
- [CastFSR: A Fast--Slow--Reflect Agentic Reasoning Framework for Context-Aware Time Series Forecasting](https://mlsanity.com/paper/2608.03031/) (2026)
- [CoGenCast: A Coupled Autoregressive-Flow Generative Framework for Time Series Forecasting](https://mlsanity.com/paper/2602.03564/) (2026)
- [Time Series as Language: A Universal Tokenizer for General-Purpose Time Series Foundation Models](https://mlsanity.com/paper/2606.09861/) (2026)
- [A Human-in-the-Loop Autonomous Agent for Industry Time Series Forecasting](https://mlsanity.com/paper/2608.30976/) (2026)

## Abstract (arXiv)

Time series forecasting plays a vital role in supporting decision-making across a wide range of critical applications, including energy, healthcare, and finance. Despite recent advances, forecasting accuracy remains limited due to the challenge of integrating historical numerical sequences with contextual features, which often comprise unstructured textual data. To address this challenge, we propose TokenCast, a large language model (LLM) driven framework that leverages language-based symbolic representations as a unified intermediary for context-aware time series forecasting. Specifically, TokenCast employs a discrete tokenizer to transform continuous numerical sequences into temporal tokens, enabling structural alignment with language-based inputs. To effectively bridge the semantic gap between modalities, both temporal and contextual tokens are embedded into a shared representation space via a pre-trained LLM, further optimized with generative objectives. Building upon this unified semantic space, the aligned LLM is subsequently fine-tuned in a supervised manner to predict future temporal tokens, which are then decoded back into the original numerical space. Extensive experiments on real-world datasets demonstrate the effectiveness of our framework and highlight its potential as a generative framework for context-aware time series forecasting. The code is available at https://github.com/Xiaoyu-Tao/TokenCast.