Personalized Federated Sparse Adaptation of Time-Series Foundation Models
On the map: Mixture-of-experts → MoE for time series and graphs
Abstract (arXiv)
Federated adaptation of time-series foundation models (TSFMs) is attractive for building energy forecasting because meter data are private, distributed, and highly non-IID. However, a single parameter-sharing strategy is unlikely to serve all pretrained TSFMs or building clients: fully shared adapters can suppress building-specific temporal behavior, while fully local adaptation discards cross-building transfer. We propose a personalized federated sparse adaptation framework with a heterogeneous temporal mixture-of-experts (MoE) adapter placed after the pretrained TSFM representation. A sequence-level router maps each 168-hour context window to a top-k subset of experts specialized for periodicity, long-range interactions, local variation, trend-residual structure, and multi-resolution behavior. We compare global FL, local training, and personalized FL variants with globally shared or client-private expert banks. Across 50 buildings and three TSFM backbones, personalization consistently outperforms Global FL-MoE and Local MoE, while the best sparse-adaptation strategy varies by backbone and metric. Routing behavior further reveals client-level expert specialization, expert concentration, and near-uniform routing across backbones, showing that federated TSFM adaptation should be both client-aware and backbone-aware.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MoE for time series and graphs, one of 12 directions in Mixture-of-experts. Builds on 17 of its 26 references that are in the map. First posted on arXiv 5 Aug 2026.
Builds on
17 of 26 references.
Closest ancestors · 1
Same direction, newest first
Other references · 16
- 2017Attention Is All You Needcited by 60,361
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2018Federated Optimization in Heterogeneous Networkscited by 2,987
- 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
- 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
- 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
- 2020Personalized Federated Learning with Moreau Envelopescited by 618
- 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
- 2019Federated Learning with Personalization Layerscited by 541
- 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
4 more
- 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 182
- 2024TSFM-Bench: A Comprehensive and Unified Benchmark of Foundation Models for Time Series Forecastingcited by 58
- 2024Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecastingcited by 26
- 2024Federated Foundation Models on Heterogeneous Time Seriescited by 13
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Fed-GAME: Personalized Federated Learning with Graph Attention Mixture-of-Experts For Time-Series Forecasting
- 2026AME-TS: Anchored Mixture-of-Experts for Time Series Forecasting
- 2025Time Tracker: Mixture-of-Experts-Enhanced Foundation Time Series Forecasting Model with Decoupled Training Pipelines
- 2026One-Shot Price Forecasting with Covariate-Guided Experts under Privacy Constraints
- 2026QuantFlow: A Federated Mamba-Based Post-Transformer Foundation Model for Time-Series Forecasting
- 2025MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models
- 2026UB-SMoE: Universally Balanced Sparse Mixture-of-Experts for Resource-adaptive Federated Fine-tuning of Foundation Models
- 2026WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting