mlsanityData to 14 Oct 2026

Personalized Federated Sparse Adaptation of Time-Series Foundation Models

arXiv 2608.04695 ↗First posted 5 Aug 2026cs.LG cs.AI stat.MLcited by 0

On the map: Mixture-of-experts → MoE for time series and graphs

Abstract (arXiv)

Federated adaptation of time-series foundation models (TSFMs) is attractive for building energy forecasting because meter data are private, distributed, and highly non-IID. However, a single parameter-sharing strategy is unlikely to serve all pretrained TSFMs or building clients: fully shared adapters can suppress building-specific temporal behavior, while fully local adaptation discards cross-building transfer. We propose a personalized federated sparse adaptation framework with a heterogeneous temporal mixture-of-experts (MoE) adapter placed after the pretrained TSFM representation. A sequence-level router maps each 168-hour context window to a top-k subset of experts specialized for periodicity, long-range interactions, local variation, trend-residual structure, and multi-resolution behavior. We compare global FL, local training, and personalized FL variants with globally shared or client-private expert banks. Across 50 buildings and three TSFM backbones, personalization consistently outperforms Global FL-MoE and Local MoE, while the best sparse-adaptation strategy varies by backbone and metric. Routing behavior further reveals client-level expert specialization, expert concentration, and near-uniform routing across backbones, showing that federated TSFM adaptation should be both client-aware and backbone-aware.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MoE for time series and graphs, one of 12 directions in Mixture-of-experts. Builds on 17 of its 26 references that are in the map. First posted on arXiv 5 Aug 2026.

Builds on

17 of 26 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts14 Oct

Other references · 16

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  7. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  8. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  9. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  10. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  11. 2019Federated Learning with Personalization Layerscited by 541
  12. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
4 more
  1. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 182
  2. 2024TSFM-Bench: A Comprehensive and Unified Benchmark of Foundation Models for Time Series Forecastingcited by 58
  3. 2024Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecastingcited by 26
  4. 2024Federated Foundation Models on Heterogeneous Time Seriescited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Fed-GAME: Personalized Federated Learning with Graph Attention Mixture-of-Experts For Time-Series Forecasting
  2. 2026AME-TS: Anchored Mixture-of-Experts for Time Series Forecasting
  3. 2025Time Tracker: Mixture-of-Experts-Enhanced Foundation Time Series Forecasting Model with Decoupled Training Pipelines
  4. 2026One-Shot Price Forecasting with Covariate-Guided Experts under Privacy Constraints
  5. 2026QuantFlow: A Federated Mamba-Based Post-Transformer Foundation Model for Time-Series Forecasting
  6. 2025MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models
  7. 2026UB-SMoE: Universally Balanced Sparse Mixture-of-Experts for Resource-adaptive Federated Fine-tuning of Foundation Models
  8. 2026WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting