mlsanityData to 7 Oct 2026

Beyond Uniform Tokens: Adaptive Compression for Time Series Language Models

arXiv 2606.13624 ↗First posted 11 Jun 2026cs.CLcited by 0

On the map: Efficient LLM inference and serving → Prompt and context compression

Abstract (arXiv)

Large language models (LLMs) have enabled time series (TS) analysis by jointly modeling numerical observations and textual context through a shared token interface. However, TS tokens and prompt tokens exhibit fundamentally different information structures, making uniform token processing inefficient. In this paper, we study token efficiency in TS language modeling from an asymmetric-token perspective. We show that TS tokens have highly uneven spectral contributions, where many tokens share redundant frequency patterns while a small subset preserves critical temporal evidence. We also observe that prompt-token influence attenuates with model depth, suggesting that full prompt retention across all layers is unnecessary. Based on these findings, we develop an adaptive token budgeting framework that compresses TS tokens via frequency-domain structure and progressively reduces prompt tokens across layers. Experiments across forecasting, classification, imputation, and anomaly detection demonstrate up to textit7.68× inference acceleration and performance gains in textit78% of evaluated settings, showing the effectiveness of asymmetric token compression for scalable TS foundation models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving. Builds on 29 of its 53 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

29 of 53 references.

References · 29

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  4. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  5. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  6. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  7. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  8. 2022Token Merging: Your ViT But Fastercited by 995
  9. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
  10. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 606
  11. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  12. 2018Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholdingcited by 407
17 more
  1. 2021SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interactioncited by 335
  2. 2018The UEA multivariate time series classification archive, 2018cited by 330
  3. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Tokencited by 164
  4. 2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasterscited by 127
  5. 2024CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuningcited by 105
  6. 2024Large Language Models Meet NLP: A Surveycited by 102
  7. 2024TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignmentcited by 97
  8. 2025Time-MQA: Time Series Multi-Task Question Answering with Context Enhancementcited by 74
  9. 2024LSTPrompt: Large Language Models as Zero-Shot Time Series Forecasters by Long-Short-Term Promptingcited by 56
  10. 2025ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask Datasetcited by 35
  11. 2024A Picture is Worth A Thousand Numbers: Enabling LLMs Reason about Time Series via Visualizationcited by 31
  12. 2025FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecastingcited by 18
  13. 2025Time Series Analysis in Frequency Domain: A Survey of Open Challenges, Opportunities and Benchmarkscited by 10
  14. 2025Vision-Enhanced Time Series Forecasting via Latent Diffusion Modelscited by 9
  15. 2025Towards Non-Stationary Time Series Forecasting with Temporal Stabilization and Frequency Differencingcited by 8
  16. 2024Multimodal Large Language Models and Tunings: Vision, Language, Sensors, Audio, and Beyondcited by 7
  17. 2024Robust Multivariate Time Series Forecasting against Intra- and Inter-Series Transitional Shiftcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Adapting Large Language Models for Time Series Modeling via a Novel Parameter-efficient Adaptation Method
  2. 2024Understanding Why Large Language Models Can Be Ineffective in Time Series Analysis: The Impact of Modality Alignment
  3. 2025TsLLM: Augmenting LLMs for General Time Series Understanding and Prediction
  4. 2025The Few Govern the Many:Unveiling Few-Layer Dominance for Time Series Models
  5. 2024AutoTimes: Autoregressive Time Series Forecasters via Large Language Models
  6. 2025Lossless Compression of Large Language Model-Generated Text via Next-Token Prediction
  7. 2025Adapting LLMs to Time Series Forecasting via Temporal Heterogeneity Modeling and Representation Alignment
  8. 2026Rethinking the Role of LLMs in Time Series Forecasting