mlsanityData to 14 Oct 2026

Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecasting

arXiv 2405.14252 ↗First posted 23 May 2024cs.LGcited by 26

Not on the map. Why?

Abstract (arXiv)

Unlike natural language processing and computer vision, the development of Foundation Models (FMs) for time series forecasting is blocked due to data scarcity. While recent efforts are focused on building such FMs by unlocking the potential of language models (LMs) for time series analysis, dedicated parameters for various downstream forecasting tasks need training, which hinders the common knowledge sharing across domains. Moreover, data owners may hesitate to share the access to local data due to privacy concerns and copyright protection, which makes it impossible to simply construct a FM on cross-domain training instances. To address these issues, we propose Time-FFM, a Federated Foundation Model for Time series forecasting by leveraging pretrained LMs. Specifically, we begin by transforming time series into the modality of text tokens. To bootstrap LMs for time series reasoning, we propose a prompt adaption module to determine domain-customized prompts dynamically instead of artificially. Given the data heterogeneity across domains, we design a personalized federated training strategy by learning global encoders and local prediction heads. Our comprehensive experiments indicate that Time-FFM outperforms state-of-the-arts and promises effective few-shot and zero-shot forecaster.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 42 of its 57 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

42 of 57 references.

References · 42

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  7. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  8. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  9. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  10. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  11. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  12. 2020Reformer: The Efficient Transformercited by 1,668
30 more
  1. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  2. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  3. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  4. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  5. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,013
  6. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
  7. 2023A Survey on Multimodal Large Language Modelscited by 909
  8. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 879
  9. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 788
  10. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 753
  11. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  12. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  13. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  14. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  15. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
  16. 2022Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistencycited by 250
  17. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 239
  18. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 185
  19. 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 149
  20. 2023UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecastingcited by 144
  21. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 128
  22. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 110
  23. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 99
  24. 2021Voice2Series: Reprogramming Acoustic Models for Time Series Classificationcited by 91
  25. 2022When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methodscited by 75
  26. 2023Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimizationcited by 65
  27. 2022Model Reprogramming: Resource-Efficient Cross-Domain Machine Learningcited by 64
  28. 2023FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Modelscited by 63
  29. 2022Beyond Just Vision: A Review on Self-Supervised Representation Learning on Multimodal and Temporal Datacited by 27
  30. 2024Personalized Federated Instruction Tuning via Neural Architecture Searchcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 16; 2026: 8.

26 papers.

Sort:

By area:

  1. 2025Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecastingcited by 63
  2. 2025Time Series Forecasting via Reasoning: A Slow-Thinking Approach with Reinforcement Fine-Tuned LLMscited by 34
  3. 2024Federated Foundation Models on Heterogeneous Time Seriescited by 13
  4. 2025Towards Cross-Modality Modeling for Time Series Analytics: A Survey in the LLM Eracited by 12
  5. 2025Distributed LLMs and Multimodal Large Language Models: A Survey on Advances, Challenges, and Future Directionscited by 10
  6. 2025Vision-Enhanced Time Series Forecasting via Latent Diffusion Modelscited by 9
  7. 2025ARROW: An Adaptive Rollout and Routing Method for Global Weather Forecastingcited by 8
  8. 2025CC-Time: Cross-Model and Cross-Modality Time Series Forecastingcited by 6
  9. 2025AXIS: Explainable Time Series Anomaly Detection with Large Language Modelscited by 4
  10. 2024Tackling Data Heterogeneity in Federated Time Series Forecastingcited by 3
  11. 2025Large Language models for Time Series Analysis: Techniques, Applications, and Challengescited by 3
  12. 2025KANMixer: a minimal KAN-centered mixer for long-term time series forecastingcited by 3
14 more
  1. 2025Urban Computing in the Era of Large Language Modelscited by 2
  2. 2025Entropy Guided Dynamic Patch Segmentation for Time Series Transformerscited by 2
  3. 2026Learning to Factorize and Adapt: A Versatile Approach Toward Universal Spatio-Temporal Foundation Modelscited by 2
  4. 2026Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approachcited by 2
  5. 2025A Time Series Multitask Framework Integrating a Large Language Model, Pre-Trained Time Series Model, and Knowledge Graphcited by 1
  6. 2026Discrete Prototypical Memories for Federated Time Series Foundation Modelscited by 1
  7. 2025From Time Series Analysis to Question Answering: A Survey in the LLM Eracited by 0
  8. 2025KFS: KAN based adaptive Frequency Selection learning architecture for long term time series forecastingcited by 0
  9. 2025Invariant Representation Learning for Source-Free Time Series Forecasting with LLM-Centric Proxy Denoisingcited by 0
  10. 2026Deep Time-Series Forecasting in 10 Years: A Surveycited by 0
  11. 2026Beyond Tokenization: Direct Timestep Embedding and Contrastive Alignment for Time-Series Question Answeringcited by 0
  12. 2026TSSM: Triaxial State Space Model for Global Station Weather Forecasting with Temporal-Variable-Historical Modelingcited by 0
  13. 2026Personalized Federated Sparse Adaptation of Time-Series Foundation Modelscited by 0
  14. 2026STHMoE: Hypergraph-Enhanced Heterogeneous Dependency Coordination for LLM-Based Urban Traffic Data Forecastingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Small but Mighty: Enhancing Time Series Forecasting with Lightweight LLMs
  2. 2025Conversational Time Series Foundation Models: Towards Explainable and Effective Forecasting
  3. 2024AutoTimes: Autoregressive Time Series Forecasters via Large Language Models
  4. 2026Rethinking the Role of LLMs in Time Series Forecasting
  5. 2025Semantic-Enhanced Time-Series Forecasting via Large Language Models
  6. 2025Adapting LLMs to Time Series Forecasting via Temporal Heterogeneity Modeling and Representation Alignment
  7. 2026TimeSeriesExamAgent: Creating Time Series Reasoning Benchmarks at Scale
  8. 2025Fusing Large Language Models with Temporal Transformers for Time Series Forecasting