mlsanityData to 7 Oct 2026

Fine-Tuning Pre-trained Large Time Series Models for Prediction of Wind Turbine SCADA Data

arXiv 2412.00403 ↗First posted 30 Nov 2024cs.LG cs.AI cs.CEcited by 0

Not on the map. Why?

Abstract (arXiv)

The remarkable achievements of large models in the fields of natural language processing (NLP) and computer vision (CV) have sparked interest in their application to time series forecasting within industrial contexts. This paper explores the application of a pre-trained large time series model, Timer, which was initially trained on a wide range of time series data from multiple domains, in the prediction of Supervisory Control and Data Acquisition (SCADA) data collected from wind turbines. The model was fine-tuned on SCADA datasets sourced from two wind farms, which exhibited differing characteristics, and its accuracy was subsequently evaluated. Additionally, the impact of data volume was studied to evaluate the few-shot ability of the Timer. Finally, an application study on one-turbine fine-tuning for whole-plant prediction was implemented where both few-shot and cross-turbine generalization capacity is required. The results reveal that the pre-trained large model does not consistently outperform other baseline models in terms of prediction accuracy whenever the data is abundant or not, but demonstrates superior performance in the application study. This result underscores the distinctive advantages of the pre-trained large time series model in facilitating swift deployment.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 23 references that are in the map. First posted on arXiv 30 Nov 2024.

Builds on

15 of 23 references.

References · 15

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023Segment Anythingcited by 8,158
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  7. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  8. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  9. 2018BRITS: Bidirectional Recurrent Imputation for Time Seriescited by 288
  10. 2022Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizerscited by 204
  11. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 198
  12. 2024Are Language Models Actually Useful for Time Series Forecasting?cited by 163
3 more
  1. 2022What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?cited by 149
  2. 2023The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecastingcited by 100
  3. 2024Empowering Time Series Analysis with Foundation Models: A Comprehensive Surveycited by 23

Built on by

Not cited yet.