# Multi-Scale Finetuning for Encoder-based Time Series Foundation Models

arXiv: https://arxiv.org/abs/2506.14087 · HTML: https://mlsanity.com/paper/2506.14087/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 17 Jun 2025.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [Long-term Forecasting with TiDE: Time-series Dense Encoder](https://mlsanity.com/paper/2304.08424/) (2023)
- [Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts](https://mlsanity.com/paper/2409.16040/) (2024)
- [TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysis](https://mlsanity.com/paper/2410.16032/) (2024)
- [Causal-learn: Causal Discovery in Python](https://mlsanity.com/paper/2307.16405/) (2023)
- [Fine-tuning can cripple your foundation model; preserving features may be the solution](https://mlsanity.com/paper/2308.13320/) (2023)
- [Multi-resolution Time-Series Transformer for Long-term Forecasting](https://mlsanity.com/paper/2311.04147/) (2023)
- [AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting](https://mlsanity.com/paper/2502.10235/) (2025)
- [ElasTST: Towards Robust Varied-Horizon Forecasting with Elastic Time-Series Transformer](https://mlsanity.com/paper/2411.01842/) (2024)

## Built on by (most cited first, 14 of 14)

- [It's TIME: Towards the Next Generation of Time Series Forecasting Benchmarks](https://mlsanity.com/paper/2602.12147/) (2026)
- [STAR: Boosting Time Series Foundation Models for Anomaly Detection through State-aware Adapter](https://mlsanity.com/paper/2510.16014/) (2025)
- [TimeRFT: Stimulating Generalizable Time Series Forecasting for TSFMs via Reinforcement Finetuning](https://mlsanity.com/paper/2605.00015/) (2026)
- [Post-Training in Time Series Foundation Models: A Unifying Framework](https://mlsanity.com/paper/2607.20002/) (2026)
- [Shapelets-Enriched Selective Forecasting using Time Series Foundation Models](https://mlsanity.com/paper/2601.11821/) (2026)
- [Ground-Truth Neighborhood Regularization for Reinforcement Learning Post-Training of Time Series Foundation Models](https://mlsanity.com/paper/2608.08010/) (2026)
- [PEFT-MuTS: A Multivariate Parameter-Efficient Fine-Tuning Framework for Remaining Useful Life Prediction based on Cross-domain Time Series Representation Model](https://mlsanity.com/paper/2601.22631/) (2026)
- [Deep Time-Series Forecasting in 10 Years: A Survey](https://mlsanity.com/paper/2603.19899/) (2026)
- [Dirichlet-Guided Group Forecasting for Alleviating Over-smoothing in Time Series Forecasting](https://mlsanity.com/paper/2606.10592/) (2026)
- [CauseCollab: Causal Unified and Modality-Agnostic Network for Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2609.03818/) (2026)
- [SIFT: Enhancing Time Series Foundation Models via Semantic Invariance and Structural Fidelity Fine-Tuning](https://mlsanity.com/paper/2609.32676/) (2026)
- [Instance-Adaptive Prompts as Context for Time-Series Foundation Models](https://mlsanity.com/paper/2609.34786/) (2026)
- [Raw-Routed Mixture of Adapters: A Causal Intervention for Routing Collapse in Time Series Foundation Models](https://mlsanity.com/paper/2609.39445/) (2026)
- [IND-Eval: Aligning Evaluation and Simulation for Industrial Time Series Foundation Models](https://mlsanity.com/paper/2610.16652/) (2026)

## Abstract (arXiv)

Time series foundation models (TSFMs) demonstrate impressive zero-shot performance for time series forecasting. However, an important yet underexplored challenge is how to effectively finetune TSFMs on specific downstream tasks. While naive finetuning can yield performance gains, we argue that it falls short of fully leveraging TSFMs' capabilities, often resulting in overfitting and suboptimal performance. Given the diverse temporal patterns across sampling scales and the inherent multi-scale forecasting capabilities of TSFMs, we adopt a causal perspective to analyze finetuning process, through which we highlight the critical importance of explicitly modeling multiple scales and reveal the shortcomings of naive approaches. Focusing on encoder-based TSFMs, we propose Multiscale finetuning (MSFT), a simple yet general framework that explicitly integrates multi-scale modeling into the finetuning process. Experimental results on three different backbones (Moirai, Moment and Units) demonstrate that TSFMs finetuned with MSFT not only outperform naive and typical parameter efficient finetuning methods but also surpass state-of-the-art deep learning methods. Codes are available at https://github.com/zqiao11/MSFT.