mlsanityData to 14 Oct 2026

FedHCA2: Towards Hetero-Client Federated Multi-Task Learning

arXiv 2311.13250 ↗First posted 22 Nov 2023cs.CV cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) enables joint training across distributed clients using their local data privately. Federated Multi-Task Learning (FMTL) builds on FL to handle multiple tasks, assuming model congruity that identical model architecture is deployed in each client. To relax this assumption and thus extend real-world applicability, we introduce a novel problem setting, Hetero-Client Federated Multi-Task Learning (HC-FMTL), to accommodate diverse task setups. The main challenge of HC-FMTL is the model incongruity issue that invalidates conventional aggregation methods. It also escalates the difficulties in accurate model aggregation to deal with data and task heterogeneity inherent in FMTL. To address these challenges, we propose the FedHCA2 framework, which allows for federated training of personalized models by modeling relationships among heterogeneous clients. Drawing on our theoretical insights into the difference between multi-task and federated optimization, we propose the Hyper Conflict-Averse Aggregation scheme to mitigate conflicts during encoder updates. Additionally, inspired by task interaction in MTL, the Hyper Cross Attention Aggregation scheme uses layer-wise cross attention to enhance decoder interactions while alleviating model incongruity. Moreover, we employ learnable Hyper Aggregation Weights for each client to customize personalized parameter updates. Extensive experiments demonstrate the superior performance of FedHCA2 in various HC-FMTL scenarios compared to representative methods. Our code will be made publicly available.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 69 of its 97 references that are in the map. First posted on arXiv 22 Nov 2023.

Builds on

69 of 97 references.

References · 69

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  5. 2019Advances and Open Problems in Federated Learningcited by 3,046
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  7. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  8. 2021Vision Transformers for Dense Predictioncited by 1,803
  9. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  10. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  11. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  12. 2022Planning-oriented Autonomous Drivingcited by 1,087
38 more
  1. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  2. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  3. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  4. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  5. 2021Model-Contrastive Federated Learningcited by 766
  6. 2018How To Backdoor Federated Learningcited by 765
  7. 2017Federated Multi-Task Learningcited by 672
  8. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  9. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  10. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  11. 2020Federated Learning with Matched Averagingcited by 564
  12. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  13. 2019Federated Learning with Personalization Layerscited by 541
  14. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  15. 2018End-to-End Multi-Task Learning with Attentioncited by 534
  16. 2016Cross-stitch Networks for Multi-task Learningcited by 467
  17. 2021Towards Personalized Federated Learningcited by 464
  18. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  19. 2020Balanced Meta-Softmax for Long-Tailed Visual Recognitioncited by 400
  20. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  21. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  22. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 375
  23. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  24. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 283
  25. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  26. 2020Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropoutcited by 201
  27. 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
  28. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 195
  29. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 192
  30. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 182
  31. 2020Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Modelscited by 169
  32. 2020MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learningcited by 157
  33. 2019Attentive Single-Tasking of Multiple Taskscited by 148
  34. 2019Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentationcited by 142
  35. 2018NDDR-CNN: Layerwise Feature Fusing in Multi-Task CNNs by Neural Discriminative Dimensionality Reductioncited by 127
  36. 2020Learning to Branch for Multi-Task Learningcited by 115
  37. 2022Vertical Federated Learning: Concepts, Advances and Challengescited by 108
  38. 2022Layer-wised Model Aggregation for Personalized Federated Learningcited by 96

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 8; 2026: 7.

17 papers.

Sort:

By area:

  1. 2025FedVLA: Federated Vision-Language-Action Learning with Dual Gating Mixture-of-Experts for Robotic Manipulationcited by 18
  2. 2024Feasibility of Federated Learning from Client Databases with Different Brain Diseases and MRI Modalitiescited by 5
  3. 2024Federated Communication-Efficient Multi-Objective Optimizationcited by 3
  4. 2025pMixFed: Efficient Personalized Federated Learning through Adaptive Layer-Wise Mixupcited by 3
  5. 2025Redefining non-IID Data in Federated Learning for Computer Vision Tasks: Migrating from Labels to Embeddings for Task-Specific Data Distributionscited by 3
  6. 2026TextResNet: Decoupling and Routing Optimization Signals in Compound AI Systems via Deep Residual Tuningcited by 3
  7. 2025Few-shot Implicit Function Generation via Equivariancecited by 2
  8. 2025TAP: Two-Stage Adaptive Personalization of Multi-Task and Multi-Modal Foundation Models in Federated Learningcited by 2
  9. 2026Fed-ADE: Adaptive Learning Rate for Federated Post-adaptation under Distribution Shiftcited by 2
  10. 2025Towards Unified Modeling in Federated Multi-Task Learning via Subspace Decouplingcited by 1
  11. 2025Helmsman: Autonomous Synthesis of Federated Learning Systems via Collaborative LLM Agentscited by 1
  12. 2026Semantic-Aware Task Clustering for Federated Cooperative Multi-Task Semantic Communicationcited by 1
5 more
  1. 2025ColNet: Collaborative Optimization in Decentralized Federated Multi-task Learning Systemscited by 0
  2. 2026SurgFed: Language-guided Multi-Task Federated Learning for Surgical Video Understandingcited by 0
  3. 2026Bidirectional Learning of Facial Action Units and Expressions via Structured Semantic Mapping across Heterogeneous Datasetscited by 0
  4. 2026Rethinking Air-Ground Collaboration: A Progressive Cross-Task Benchmark and Socialized Learning Frameworkcited by 0
  5. 2026Decentralized Federated Learning for Heterogeneous Multi-Task Semantic Communicationcited by 0