# NeST: Neighborhood-aware semantic alignment and temporal modulation for LLM based time series forecasting

arXiv: https://arxiv.org/abs/2412.04806 · HTML: https://mlsanity.com/paper/2412.04806/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 42 references that are in the map.
- First posted on arXiv 6 Dec 2024.

## Builds on (most cited first, 16 of 16)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks](https://mlsanity.com/paper/2005.11650/) (2020)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [TabLLM: Few-shot Classification of Tabular Data with Large Language Models](https://mlsanity.com/paper/2210.10723/) (2022)
- [TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting](https://mlsanity.com/paper/2310.04948/) (2023)
- [MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokens](https://mlsanity.com/paper/2404.03413/) (2024)
- [Tuning LayerNorm in Attention: Towards Efficient Multi-Modal LLM Finetuning](https://mlsanity.com/paper/2312.11420/) (2023)
- [Empowering Time Series Analysis with Large Language Models: A Survey](https://mlsanity.com/paper/2402.03182/) (2024)
- [RobustSTL: A Robust Seasonal-Trend Decomposition Algorithm for Long Time Series](https://mlsanity.com/paper/1812.01767/) (2018)

## Built on by (most cited first, 1 of 1)

- [A Survey of Reasoning and Agentic Systems in Time Series with Large Language Models](https://mlsanity.com/paper/2509.11575/) (2025)

## Abstract (arXiv)

Adapting Large Language Models (LLMs) trained on discrete text data, to forecast continuous time series signals is challenging. While finetuning the LLMs enables such adaptation, effectively integrating both textual and time series information in the prompt is critical. Current LLM-based time series forecasting methods combine the two modalities through simple concatenation or parameter heavy cross-attention. Moreover, existing methods embed time series data using decomposition techniques that may inadequately capture complex temporal dynamics. To address these limitations, we propose neighborhood-aware semantic alignment and temporal modulation based framework (NEST) to formulate a new text-integrated time series prompt to finetune the LLM. First, we generate neighborhood-aware text prototypes that are optimized to represent local neighborhoods of pretrained word token embeddings of the LLM. Second, we align them with temporal representations of the time series input using a nearest-neighbor contrastive objective, after which the top-k most relevant text prototypes are retrieved. Third, we introduce text prototype conditioned temporal modulation that uses the retrieved text prototypes to adaptively scale and shift time series features. Extensive experiments demonstrate that NeST consistently outperforms state-of-the-art methods across eight benchmarks, achieving an average 1.2% reduction in MSE for long-term forecasting. In addition, it demonstrates strong generalization, yielding an average 4.9% reduction in MSE in zero-shot forecasting. Beyond benchmark datasets, NeST also delivers robust performance on a real-world distributed photovoltaic power forecasting task across nine datasets, improving the average R2 score by 3.3%. These findings demonstrate the effectiveness and generalizability of NeST for adapting LLMs to diverse time series forecasting tasks.