mlsanityData to 14 Oct 2026

FedMMKT:Co-Enhancing a Server Text-to-Image Model and Client Task Models in Multi-Modal Federated Learning

arXiv 2510.12254 ↗First posted 14 Oct 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Text-to-Image (T2I) models have demonstrated their versatility in a wide range of applications. However, adaptation of T2I models to specialized tasks is often limited by the availability of task-specific data due to privacy concerns. On the other hand, harnessing the power of rich multimodal data from modern mobile systems and IoT infrastructures presents a great opportunity. This paper introduces Federated Multi-modal Knowledge Transfer (FedMMKT), a novel framework that enables co-enhancement of a server T2I model and client task-specific models using decentralized multimodal data without compromising data privacy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 53 of its 75 references that are in the map. First posted on arXiv 14 Oct 2025.

Builds on

53 of 75 references.

References · 53

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  8. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  9. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  11. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  12. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
38 more
  1. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  3. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2016Improved Techniques for Training GANscited by 2,970
  6. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  7. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  8. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  9. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  10. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  11. 2023Aligning Text-to-Image Models using Human Feedbackcited by 422
  12. 2022Multimodal Learning with Transformers: A Surveycited by 400
  13. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 388
  14. 2022Pretraining is All You Need for Image-to-Image Translationcited by 160
  15. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  16. 2022PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Modelcited by 134
  17. 2023SLoRA: Federated Parameter Efficient Fine-Tuning of Language Modelscited by 93
  18. 2024Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resourcescited by 93
  19. 2023Multimodal Federated Learning via Contrastive Representation Ensemblecited by 86
  20. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 84
  21. 2022Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learningcited by 81
  22. 2023Offsite-Tuning: Transfer Learning without Full Modelcited by 78
  23. 2022When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methodscited by 75
  24. 2018How good is my GAN?cited by 74
  25. 2024FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Modelcited by 74
  26. 2024Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigationcited by 66
  27. 2023FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learningcited by 60
  28. 2023DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedbackcited by 58
  29. 2023FedMultimodal: A Benchmark For Multimodal Federated Learningcited by 51
  30. 2021FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusioncited by 49
  31. 2024DiffusionAgent: Navigating Expert Models for Agentic Image Generationcited by 28
  32. 2024FedMKT: Federated Mutual Knowledge Transfer for Large and Small Language Modelscited by 19
  33. 2023FedMEKT: Distillation-based Embedding Knowledge Transfer for Multimodal Federated Learningcited by 17
  34. 2024An Upload-Efficient Scheme for Transferring Knowledge From a Server-Side Pre-trained Generator to Clients in Heterogeneous Federated Learningcited by 17
  35. 2024FedPFT: Federated Proxy Fine-Tuning of Foundation Modelscited by 16
  36. 2023DomainStudio: Fine-Tuning Diffusion Models for Domain-Driven Image Generation using Limited Datacited by 15
  37. 2023Mutual Enhancement of Large and Small Language Models with Cross-Silo Knowledge Transfercited by 13
  38. 2024Towards Multi-modal Transformers in Federated Learningcited by 12

50 most cited of 53 All 53 (JSON)

Built on by

Not cited yet.