# TimeGPT in Load Forecasting: A Large Time Series Model Perspective

arXiv: https://arxiv.org/abs/2404.04885 · HTML: https://mlsanity.com/paper/2404.04885/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 47 references that are in the map.
- First posted on arXiv 7 Apr 2024.

## Builds on (most cited first, 8 of 8)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [TimeGPT-1](https://mlsanity.com/paper/2310.03589/) (2023)
- [ES-dRNN: A Hybrid Exponential Smoothing and Dilated Recurrent Neural Network Model for Short-Term Load Forecasting](https://mlsanity.com/paper/2112.02663/) (2021)
- [Scenario Generation for Cooling, Heating, and Power Loads Using Generative Moment Matching Networks](https://mlsanity.com/paper/2102.03360/) (2021)

## Built on by (most cited first, 8 of 8)

- [A Survey of AIOps in the Era of Large Language Models](https://mlsanity.com/paper/2507.12472/) (2025)
- [Short-Term Electricity-Load Forecasting by Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2408.16202/) (2024)
- [Decision-Focused Fine-Tuning of Time Series Foundation Models for Dispatchable Feeder Optimization](https://mlsanity.com/paper/2503.01936/) (2025)
- [RuntimeSlicer: Towards Generalizable Unified Runtime State Representation for Failure Management](https://mlsanity.com/paper/2603.21495/) (2026)
- [Probabilistic Forecasting for Building Energy Systems using Time-Series Foundation Models](https://mlsanity.com/paper/2506.00630/) (2025)
- [Large Language Model-Empowered Interactive Load Forecasting](https://mlsanity.com/paper/2505.16577/) (2025)
- [Leveraging Time-Series Foundation Model for Subsurface Well Logs Prediction and Anomaly Detection](https://mlsanity.com/paper/2412.05681/) (2024)
- [QKCV Attention: Enhancing Time Series Forecasting with Static Categorical Embeddings for Both Lightweight and Pre-trained Foundation Models](https://mlsanity.com/paper/2510.20222/) (2025)

## Abstract (arXiv)

Machine learning models have made significant progress in load forecasting, but their forecast accuracy is limited in cases where historical load data is scarce. Inspired by the outstanding performance of large language models (LLMs) in computer vision and natural language processing, this paper aims to discuss the potential of large time series models in load forecasting with scarce historical data. Specifically, the large time series model is constructed as a time series generative pre-trained transformer (TimeGPT), which is trained on massive and diverse time series datasets consisting of 100 billion data points (e.g., finance, transportation, banking, web traffic, weather, energy, healthcare, etc.). Then, the scarce historical load data is used to fine-tune the TimeGPT, which helps it to adapt to the data distribution and characteristics associated with load forecasting. Simulation results show that TimeGPT outperforms the benchmarks (e.g., popular machine learning models and statistical models) for load forecasting on several real datasets with scarce training samples, particularly for short look-ahead times. However, it cannot be guaranteed that TimeGPT is always superior to benchmarks for load forecasting with scarce data, since the performance of TimeGPT may be affected by the distribution differences between the load data and the training data. In practical applications, we can divide the historical data into a training set and a validation set, and then use the validation set loss to decide whether TimeGPT is the best choice for a specific dataset.