mlsanityData to 14 Oct 2026

FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction

arXiv 2407.19389 ↗First posted 28 Jul 2024cs.DC cs.LG math.OCcited by 21

Not on the map. Why?

Abstract (arXiv)

In federated learning (FL), accommodating clients' varied computational capacities poses a challenge, often limiting the participation of those with constrained resources in global model training. To address this issue, the concept of model heterogeneity through submodel extraction has emerged, offering a tailored solution that aligns the model's complexity with each client's computational capacity. In this work, we propose Federated Importance-Aware Submodel Extraction (FIARSE), a novel approach that dynamically adjusts submodels based on the importance of model parameters, thereby overcoming the limitations of previous static and dynamic submodel extraction methods. Compared to existing works, the proposed method offers a theoretical foundation for the submodel extraction and eliminates the need for additional information beyond the model parameters themselves to determine parameter importance, significantly reducing the overhead on clients. Extensive experiments are conducted on various datasets to showcase the superior performance of the proposed FIARSE.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 58 of its 83 references that are in the map. First posted on arXiv 28 Jul 2024.

Builds on

58 of 83 references.

References · 58

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  5. 2019Advances and Open Problems in Federated Learningcited by 3,046
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  8. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  9. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  10. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  11. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  12. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
38 more
  1. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  2. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  3. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  4. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 527
  5. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  6. 2021A Field Guide to Federated Optimizationcited by 315
  7. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 222
  8. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
  9. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  10. 2021Does Knowledge Distillation Really Work?cited by 190
  11. 2019Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterizationcited by 189
  12. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  13. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 163
  14. 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
  15. 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 123
  16. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 109
  17. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  18. 2020On the Convergence of SGD with Biased Gradientscited by 83
  19. 2022Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learningcited by 81
  20. 2021Top-KAST: Top-K Always Sparse Trainingcited by 79
  21. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 79
  22. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  23. 2024FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Modelcited by 74
  24. 2023FedExP: Speeding Up Federated Averaging via Extrapolationcited by 69
  25. 2020Cost-Effective Federated Learning Designcited by 62
  26. 2020LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasetscited by 60
  27. 2022Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learningcited by 59
  28. 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 55
  29. 2021How Well Do Sparse Imagenet Models Transfer?cited by 50
  30. 2022FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clientscited by 46
  31. 2021Anarchic Federated Learningcited by 42
  32. 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
  33. 2022Sparse Random Networks for Communication-Efficient Federated Learningcited by 32
  34. 2023Towards Personalized Federated Learning via Heterogeneous Model Reassemblycited by 31
  35. 2021Effective Model Sparsification by Scheduled Grow-and-Prune Methodscited by 26
  36. 2023Every Parameter Matters: Ensuring the Convergence of Federated Learning with Dynamic Heterogeneous Models Reductioncited by 24
  37. 2022Anchor Sampling for Federated Learning with Partial Client Participationcited by 14
  38. 2023Is Aggregation the Only Choice? Federated Learning via Layer-wise Model Recombinationcited by 13

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 10; 2026: 10.

21 papers.

Sort:

  1. 2026MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learningcited by 11
  2. 2025FedEL: Federated Elastic Learning for Heterogeneous Devicescited by 3
  3. 2024HeteroTune: Efficient Federated Learning for Large Heterogeneous Modelscited by 2
  4. 2025FedFetch: Faster Federated Learning with Adaptive Downstream Prefetchingcited by 2
  5. 2025FedADP: Unified Model Aggregation for Federated Learning with Heterogeneous Model Architecturescited by 2
  6. 2025C^2Prompt: Class-aware Client Knowledge Interaction for Federated Continual Learningcited by 2
  7. 2025On the Convergence and Stability of Distributed Sub-model Trainingcited by 1
  8. 2025CycleSL: Server-Client Cyclical Update Driven Scalable Split Learningcited by 1
  9. 2025FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learningcited by 1
  10. 2025Energy and Memory-Efficient Federated Learning With Ordered Layer Freezingcited by 1
  11. 2026Towards Robust Federated Multimodal Graph Learning under Modality Heterogeneitycited by 1
  12. 2025Towards Privacy-Preserving and Heterogeneity-aware Split Federated Learning via Probabilistic Maskingcited by 0
9 more
  1. 2025UnifiedFL: A Dynamic Unified Learning Framework for Equitable Federationcited by 0
  2. 2026Representation-Aligned Multi-Scale Personalization for Federated Learningcited by 0
  3. 2026A Taxonomy and Resolution Strategy for Client-Level Disagreements in Federated Learningcited by 0
  4. 2026From Coordinate Matching to Structural Alignment: Rethinking Prototype Alignment in Heterogeneous Federated Learningcited by 0
  5. 2026HARMONY: Bridging the Personalization-Generalization Gap by Mitigating Representation Skew in Heterogeneous Split Federated Learningcited by 0
  6. 2026Capacity Confounds and Coverage Guarantees in Adaptive Sub-model Federated Learningcited by 0
  7. 2026FEAST: Federated Shared-Space Training for Resource-Heterogeneous Clientscited by 0
  8. 2026FedFIbOS: Fisher Importance based Optimal Submodelling for Heterogeneous Federated Learningcited by 0
  9. 2026FedSAP: Federated Learning with Structured Adaptive Partitioning for Multi-Domain Heterogeneous Edge Devicescited by 0