mlsanityData to 7 Oct 2026

FedSocket: Recipient-Executable Knowledge Exchange for Heterogeneous Multimodal Federated Learning

arXiv 2609.37582 ↗First posted 29 Sep 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated knowledge must remain usable by recipients with different modalities, private architectures, and tasks. We present FedSocket, which makes recipient execution a design requirement of the exchanged model. A shared Q combines recipient-computable inputs, task-owned outputs, and ownership-aware aggregation, connecting heterogeneous private models through a common prediction interface. Private models teach local Q copies; the returned Q supports local learning and Joint inference, with only Q parameters and counts exchanged. Across six datasets, FedSocket improves missing-modality recipient accuracy over Local by 14.44 and 15.51 percentage points on MELD and UCF-51. Under matched inference capacity, Joint exceeds independent ensembles by 11.06 points in UCF-51 accuracy and 4.87 points in mean bidirectional Flickr30k R@1. Joint also improves over Q alone on all four heterogeneous endpoints, demonstrating the value of combining local and exchanged predictions. Teacher controls, sharing-path interventions, and component factorials identify the roles of supervision, sharing, and deployment. FedSocket makes exchanged knowledge directly usable from federated training to recipient inference.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 30 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

21 of 30 references.

References · 21

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  5. 2019Contrastive Representation Distillationcited by 734
  6. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  7. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  8. 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 259
  9. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 185
  10. 2022PMR: Prototypical Modal Rebalance for Multimodal Learningcited by 114
  11. 2023Multimodal Federated Learning via Contrastive Representation Ensemblecited by 85
  12. 2022Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learningcited by 81
9 more
  1. 2020Federated Mutual Learningcited by 70
  2. 2023FedMultimodal: A Benchmark For Multimodal Federated Learningcited by 51
  3. 2021FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusioncited by 49
  4. 2024MH-pFLID: Model Heterogeneous personalized Federated Learning via Injection and Distillation for Medical Data Analysiscited by 14
  5. 2023Enhanced Multimodal Representation Learning with Cross-modal KDcited by 11
  6. 2023Overcome Modal Bias in Multi-modal Federated Learning via Balanced Modality Selectioncited by 6
  7. 2026Federated Prompt-Tuning with Heterogeneous and Incomplete Multimodal Client Datacited by 3
  8. 2025TACTFL: Temporal Contrastive Training for Multi-modal Federated Learning with Similarity-guided Model Aggregationcited by 1
  9. 2026FedAFD: Multimodal Federated Learning via Adversarial Fusion and Distillationcited by 1

Built on by

Not cited yet.