# Low-Rank Adaptation of Time Series Foundational Models for Out-of-Domain Modality Forecasting

arXiv: https://arxiv.org/abs/2405.10216 · HTML: https://mlsanity.com/paper/2405.10216/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 29 references that are in the map.
- First posted on arXiv 16 May 2024.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models](https://mlsanity.com/paper/2106.10199/) (2021)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [AdapterDrop: On the Efficiency of Adapters in Transformers](https://mlsanity.com/paper/2010.11918/) (2020)
- [Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2004.03829/) (2020)
- [Foundational Models Defining a New Era in Vision: A Survey and Outlook](https://mlsanity.com/paper/2307.13721/) (2023)
- [Three things everyone should know about Vision Transformers](https://mlsanity.com/paper/2203.09795/) (2022)
- [Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunity](https://mlsanity.com/paper/2305.08252/) (2023)
- [Strong Baselines for Parameter Efficient Few-Shot Fine-tuning](https://mlsanity.com/paper/2304.01917/) (2023)
- [A Transformer Architecture for Stress Detection from ECG](https://mlsanity.com/paper/2108.09737/) (2021)
- [AttX: Attentive Cross-Connections for Fusion of Wearable Signals in Emotion Recognition](https://mlsanity.com/paper/2206.04625/) (2022)
- [Interpreting Forecasted Vital Signs Using N-BEATS in Sepsis Patients](https://mlsanity.com/paper/2306.14016/) (2023)
- [Vital Sign Forecasting for Sepsis Patients in ICUs](https://mlsanity.com/paper/2311.04770/) (2023)
- [An Analysis of Physiological and Psychological Responses in Virtual Reality and Flat Screen Gaming](https://mlsanity.com/paper/2306.09690/) (2023)

## Built on by (most cited first, 18 of 18)

- [Low-Rank Adaptation for Foundation Models: A Comprehensive Review](https://mlsanity.com/paper/2501.00365/) (2024)
- [PDE-Transformer: Efficient and Versatile Transformers for Physics Simulations](https://mlsanity.com/paper/2505.24717/) (2025)
- [Zero-shot forecasting of chaotic systems](https://mlsanity.com/paper/2409.15771/) (2024)
- [Are Large Brainwave Foundation Models Capable Yet? Insights from Fine-tuning](https://mlsanity.com/paper/2507.01196/) (2025)
- [Accurate Parameter-Efficient Test-Time Adaptation for Time Series Forecasting](https://mlsanity.com/paper/2506.23424/) (2025)
- [Beyond LoRA: Exploring Efficient Fine-Tuning Techniques for Time Series Foundational Models](https://mlsanity.com/paper/2409.11302/) (2024)
- [TS-Memory: Plug-and-Play Memory for Time Series Foundation Models](https://mlsanity.com/paper/2602.11550/) (2026)
- [CALM: A Framework for Continuous, Adaptive, and LLM-Mediated Anomaly Detection in Time-Series Streams](https://mlsanity.com/paper/2508.21273/) (2025)
- [Semantic-Enhanced Time-Series Forecasting via Large Language Models](https://mlsanity.com/paper/2508.07697/) (2025)
- [Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Survey](https://mlsanity.com/paper/2601.03181/) (2026)
- [Adapting Time Series Foundation Models through Data Mixtures](https://mlsanity.com/paper/2603.02840/) (2026)
- [Mixture of Low Rank Adaptation with Partial Parameter Sharing for Time Series Forecasting](https://mlsanity.com/paper/2505.17872/) (2025)
- [Time Series Foundation Models for Process Model Forecasting](https://mlsanity.com/paper/2512.07624/) (2025)
- [Parameter-Efficient Fine-Tuning for HAR: Integrating LoRA and QLoRA into Transformer Models](https://mlsanity.com/paper/2512.17983/) (2025)
- [SPaRSe-TIME: Saliency-Projected Low-Rank Temporal Modeling for Efficient and Interpretable Time Series Prediction](https://mlsanity.com/paper/2604.17350/) (2026)
- [GlucoFM-Bench: Benchmarking Time-Series Foundation Models for Blood Glucose Forecasting](https://mlsanity.com/paper/2606.06881/) (2026)
- [Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting](https://mlsanity.com/paper/2607.22299/) (2026)
- [Volatility-Clustering Adaptation for Financial Time Series](https://mlsanity.com/paper/2609.37715/) (2026)

## Abstract (arXiv)

Low-Rank Adaptation (LoRA) is a widely used technique for fine-tuning large pre-trained or foundational models across different modalities and tasks. However, its application to time series data, particularly within foundational models, remains underexplored. This paper examines the impact of LoRA on contemporary time series foundational models: Lag-Llama, MOIRAI, and Chronos. We demonstrate LoRA's fine-tuning potential for forecasting the vital signs of sepsis patients in intensive care units (ICUs), emphasizing the models' adaptability to previously unseen, out-of-domain modalities. Integrating LoRA aims to enhance forecasting performance while reducing inefficiencies associated with fine-tuning large models on limited domain-specific data. Our experiments show that LoRA fine-tuning of time series foundational models significantly improves forecasting, achieving results comparable to state-of-the-art models trained from scratch on similar modalities. We conduct comprehensive ablation studies to demonstrate the trade-offs between the number of tunable parameters and forecasting performance and assess the impact of varying LoRA matrix ranks on model performance.