NeST: Neighborhood-aware semantic alignment and temporal modulation for LLM based time series forecasting

Adapting Large Language Models (LLMs) trained on discrete text data, to forecast continuous time series signals is challenging. While finetuning the LLMs enables such adaptation, effectively integrating both textual and time series information in the prompt is critical. Current LLM-based time series forecasting methods combine the two modalities through simple concatenation or parameter heavy cross-attention. Moreover, existing methods embed time series data using decomposition techniques that may inadequately capture complex temporal dynamics. To address these limitations, we propose neighborhood-aware semantic alignment and temporal modulation based framework (NEST) to formulate a new text-integrated time series prompt to finetune the LLM. First, we generate neighborhood-aware text prototypes that are optimized to represent local neighborhoods of pretrained word token embeddings of the LLM. Second, we align them with temporal representations of the time series input using a nearest-neighbor contrastive objective, after which the top-k most relevant text prototypes are retrieved. Third, we introduce text prototype conditioned temporal modulation that uses the retrieved text prototypes to adaptively scale and shift time series features. Extensive experiments demonstrate that NeST consistently outperforms state-of-the-art methods across eight benchmarks, achieving an average 1.2% reduction in MSE for long-term forecasting. In addition, it demonstrates strong generalization, yielding an average 4.9% reduction in MSE in zero-shot forecasting. Beyond benchmark datasets, NeST also delivers robust performance on a real-world distributed photovoltaic power forecasting task across nine datasets, improving the average R2 score by 3.3%. These findings demonstrate the effectiveness and generalizability of NeST for adapting LLMs to diverse time series forecasting tasks.