mlsanityData to 7 Oct 2026

FedX: Unsupervised Federated Learning with Cross Knowledge Distillation

arXiv 2207.09158 ↗First posted 19 Jul 2022cs.CV cs.LGcited by 40

Not on the map. Why?

Abstract (arXiv)

This paper presents FedX, an unsupervised federated learning framework. Our model learns unbiased representation from decentralized and heterogeneous local data. It employs a two-sided knowledge distillation with contrastive learning as a core component, allowing the federated system to function without requiring clients to share any data features. Furthermore, its adaptable architecture can be used as an add-on module for existing unsupervised algorithms in federated settings. Experiments show that our model improves performance significantly (1.58--5.52pp) on five unsupervised algorithms.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 36 of its 47 references that are in the map. First posted on arXiv 19 Jul 2022.

Builds on

36 of 47 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  9. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  10. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  11. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  12. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
24 more
  1. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  2. 2018Federated Learning with Non-IID Datacited by 1,099
  3. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,059
  4. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  5. 2019Relational Knowledge Distillationcited by 873
  6. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  7. 2019Towards Federated Learning at Scale: System Designcited by 826
  8. 2021Model-Contrastive Federated Learningcited by 762
  9. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 636
  10. 2020Federated Learning with Matched Averagingcited by 564
  11. 2021A Field Guide to Federated Optimizationcited by 312
  12. 2020Representation Learning via Invariant Causal Mechanismscited by 197
  13. 2018Multi-Institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentationcited by 148
  14. 2020FedVision: An Online Visual Object Detection Platform Powered by Federated Learningcited by 121
  15. 2020Self-Knowledge Distillation with Progressive Refinement of Targetscited by 106
  16. 2020Federated Unsupervised Representation Learningcited by 92
  17. 2021Collaborative Unsupervised Visual Representation Learning from Decentralized Datacited by 68
  18. 2020Improving Unsupervised Image Clustering With Robust Learningcited by 51
  19. 2020Performance Optimization for Federated Person Re-identification via Benchmark Analysiscited by 50
  20. 2021Refine Myself by Teaching Myself: Feature Refinement via Self-Knowledge Distillationcited by 46
  21. 2021Mutual Contrastive Learning for Visual Representation Learningcited by 42
  22. 2020ISD: Self-Supervised Learning by Iterative Similarity Distillationcited by 33
  23. 2021Distill on the Go: Online knowledge distillation in self-supervised learningcited by 22
  24. 2022Knowledge Sharing via Domain Adaptation in Customs Fraud Detectioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 9; 2024: 13; 2025: 11; 2026: 2.

40 papers.

Sort:

By area:

  1. 2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 137
  2. 2023Cross-Silo Prototypical Calibration for Federated Learning with Non-IID Datacited by 35
  3. 2024FedLLM-Bench: Realistic Benchmarks for Federated Learning of Large Language Modelscited by 26
  4. 2022A Hybrid Self-Supervised Learning Framework for Vertical Federated Learningcited by 17
  5. 2024Federated Distillation: A Surveycited by 14
  6. 2023Federated Learning without Full Labels: A Surveycited by 12
  7. 2023FedDefender: Client-Side Attack-Tolerant Federated Learningcited by 12
  8. 2024Rethinking the Representation in Federated Unsupervised Learning with Non-IID Datacited by 12
  9. 2023ProtoFL: Unsupervised Federated Learning via Prototypical Distillationcited by 11
  10. 2022FeDXL: Provable Federated Learning for Deep X-Risk Optimizationcited by 10
  11. 2024A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applicationscited by 8
  12. 2024Knowledge Distillation in Federated Learning: a Survey on Long Lasting Challenges and New Solutionscited by 8
28 more
  1. 2024Fast-Convergent and Communication-Alleviated Heterogeneous Hierarchical Federated Learning in Autonomous Drivingcited by 8
  2. 2023Towards Attack-tolerant Federated Learning via Critical Parameter Analysiscited by 7
  3. 2024Federated Unsupervised Domain Generalization using Global and Local Alignment of Gradientscited by 7
  4. 2024FedSC: Provable Federated Self-supervised Learning with Spectral Contrastive Objective over Non-i.i.d. Datacited by 5
  5. 2025Federated Retrieval-Augmented Generation: A Systematic Mapping Studycited by 5
  6. 2023Tackling the Non-IID Issue in Heterogeneous Federated Learning by Gradient Harmonizationcited by 4
  7. 2024CCFC: Bridging Federated Clustering and Contrastive Learningcited by 3
  8. 2025FORLA: Federated Object-centric Representation Learning with Slot Attentioncited by 3
  9. 2025FedDifRC: Unlocking the Potential of Text-to-Image Diffusion Models in Heterogeneous Federated Learningcited by 3
  10. 2025Research in Collaborative Learning Does Not Serve Cross-Silo Federated Learning in Practicecited by 3
  11. 2022Feature Correlation-guided Knowledge Transfer for Federated Self-supervised Learningcited by 1
  12. 2023Masked Autoencoders are Efficient Continual Federated Learnerscited by 1
  13. 2023UFed-GAN: A Secure Federated Learning Framework with Constrained Computation and Unlabeled Datacited by 1
  14. 2024Distributionally Robust Alignment for Medical Federated Vision-Language Pre-training Under Data Heterogeneitycited by 1
  15. 2025Hierarchical Knowledge Structuring for Effective Federated Learning in Heterogeneous Environmentscited by 1
  16. 2022Federated Unsupervised Visual Representation Learning via Exploiting General Content and Personal Stylecited by 0
  17. 2023Federated Semi-Supervised Learning with Annotation Heterogeneitycited by 0
  18. 2024FedMID: A Data-Free Method for Using Intermediate Outputs as a Defense Mechanism Against Poisoning Attacks in Federated Learningcited by 0
  19. 2024Federated Black-Box Adaptation for Semantic Segmentationcited by 0
  20. 2024Federated Deep Subspace Clusteringcited by 0
  21. 2025FedDyMem: Efficient Federated Learning with Dynamic Memory and Memory-Reduce for Unsupervised Image Anomaly Detectioncited by 0
  22. 2025Soft Separation and Distillation: Toward Global Uniformity in Federated Unsupervised Learningcited by 0
  23. 2025FedLAD: A Linear Algebra Based Data Poisoning Defence for Federated Learningcited by 0
  24. 2025FedUHD: Unsupervised Federated Learning using In-Memory Hyperdimensional Computingcited by 0
  25. 2025Cooperative Pseudo Labeling for Unsupervised Federated Classificationcited by 0
  26. 2025Federated Multi-Task Clusteringcited by 0
  27. 2026Sheaf-Based Federated Representation Learningcited by 0
  28. 2026Coordination on a Budget: Federated Active Learning with Few Labelscited by 0