mlsanityData to 7 Oct 2026

Mixture of Low Rank Adaptation with Partial Parameter Sharing for Time Series Forecasting

arXiv 2505.17872 ↗First posted 23 May 2025cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Multi-task forecasting has become the standard approach for time-series forecasting (TSF). However, we show that it suffers from an Expressiveness Bottleneck, where predictions at different time steps share the same representation, leading to unavoidable errors even with optimal representations. To address this issue, we propose a two-stage framework: first, pre-train a foundation model for one-step-ahead prediction; then, adapt it using step-specific LoRA modules.This design enables the foundation model to handle any number of forecast steps while avoiding the expressiveness bottleneck. We further introduce the Mixture-of-LoRA (MoLA) model, which employs adaptively weighted LoRA experts to achieve partial parameter sharing across steps. This approach enhances both efficiency and forecasting performance by exploiting interdependencies between forecast steps. Experiments show that MoLA significantly improves model expressiveness and outperforms state-of-the-art time-series forecasting methods. Code is available at https://anonymous.4open.science/r/MoLA-BC92.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 43 of its 54 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

43 of 54 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  5. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  6. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  7. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  8. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  9. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  10. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  11. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
  12. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
31 more
  1. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  2. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  3. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  4. 2023AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuningcited by 337
  5. 2021SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interactioncited by 335
  6. 2024LoRA Learns Less and Forgets Lesscited by 313
  7. 2021Spectral Temporal Graph Neural Network for Multivariate Time-series Forecastingcited by 298
  8. 2023LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Modelscited by 195
  9. 2023Frequency-domain MLPs are More Effective Learners in Time Series Forecastingcited by 170
  10. 2022FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecastingcited by 147
  11. 2023TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecastingcited by 136
  12. 2023FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspectivecited by 126
  13. 2023Scaling transformer neural networks for skillful and reliable medium-range weather forecastingcited by 106
  14. 2024SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameterscited by 103
  15. 2023Sparse Low-rank Adaptation of Pre-trained Language Modelscited by 96
  16. 2024Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learningcited by 80
  17. 2021Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translationcited by 74
  18. 2023Sensitivity-Aware Visual Parameter-Efficient Fine-Tuningcited by 74
  19. 2023Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inferencecited by 69
  20. 2023Parameter-Efficient Fine-Tuning without Introducing New Latencycited by 59
  21. 2023SegRNN: Segment Recurrent Neural Network for Long-Term Time Series Forecastingcited by 53
  22. 2024Fredformer: Frequency Debiased Transformer for Time Series Forecastingcited by 43
  23. 2022Multi-Scale Adaptive Graph Neural Network for Multivariate Time Series Forecastingcited by 42
  24. 2019Deep Learning for Time Series Forecasting: The Electric Load Casecited by 31
  25. 2022Efficient Fine-Tuning of BERT Models on the Edgecited by 29
  26. 2025FinTSB: A Comprehensive and Practical Benchmark for Financial Time Series Forecastingcited by 23
  27. 2024Low-Rank Adaptation of Time Series Foundational Models for Out-of-Domain Modality Forecastingcited by 18
  28. 2023MerA: Merging Pretrained Adapters For Few-Shot Learningcited by 13
  29. 2024Channel-Aware Low-Rank Adaptation in Time Series Forecastingcited by 12
  30. 2024A Note on LoRAcited by 10
  31. 2024Domain Adaptation for Time series Transformers using One-step fine-tuningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Deep Time-Series Forecasting in 10 Years: A Surveycited by 0