# L-DAWA: Layer-wise Divergence Aware Weight Aggregation in Federated Self-Supervised Visual Representation Learning

arXiv: https://arxiv.org/abs/2307.07393 · HTML: https://mlsanity.com/paper/2307.07393/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 55 references that are in the map.
- First posted on arXiv 14 Jul 2023.

## Builds on (most cited first, 30 of 44)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://mlsanity.com/paper/2203.05482/) (2022)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [Flower: A Friendly Federated Learning Research Framework](https://mlsanity.com/paper/2007.14390/) (2020)
- [Fair Resource Allocation in Federated Learning](https://mlsanity.com/paper/1905.10497/) (2019)
- [PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models](https://mlsanity.com/paper/2003.03808/) (2020)
- [SpeedNet: Learning the Speediness in Videos](https://mlsanity.com/paper/2004.06130/) (2020)
- [Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2111.14213/) (2021)
- [Parameterized Knowledge Transfer for Personalized Federated Learning](https://mlsanity.com/paper/2111.02862/) (2021)
- [Instance Localization for Self-supervised Detection Pretraining](https://mlsanity.com/paper/2102.08318/) (2021)
- [Improving Generalization in Federated Learning by Seeking Flat Minima](https://mlsanity.com/paper/2203.11834/) (2022)
- [Layer-wised Model Aggregation for Personalized Federated Learning](https://mlsanity.com/paper/2205.03993/) (2022)
- [Federated Unsupervised Representation Learning](https://mlsanity.com/paper/2010.08982/) (2020)

## Built on by (most cited first, 30 of 30)

- [LoRA-FAIR: Federated LoRA Fine-Tuning with Aggregation and Initialization Refinement](https://mlsanity.com/paper/2411.14961/) (2024)
- [Think Twice Before Selection: Federated Evidential Active Learning for Medical Image Analysis with Domain Shifts](https://mlsanity.com/paper/2312.02567/) (2023)
- [FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors](https://mlsanity.com/paper/2503.15842/) (2025)
- [Achieving Byzantine-Resilient Federated Learning via Layer-Adaptive Sparsified Model Aggregation](https://mlsanity.com/paper/2409.01435/) (2024)
- [FOOGD: Federated Collaboration for Both Out-of-distribution Generalization and Detection](https://mlsanity.com/paper/2410.11397/) (2024)
- [Rethinking the Representation in Federated Unsupervised Learning with Non-IID Data](https://mlsanity.com/paper/2403.16398/) (2024)
- [FedMLLM: Federated Fine-tuning MLLM on Multimodal Heterogeneity Data](https://mlsanity.com/paper/2411.14717/) (2024)
- [FedLWS: Federated Learning with Adaptive Layer-wise Weight Shrinking](https://mlsanity.com/paper/2503.15111/) (2025)
- [Navigating Distribution Shifts in Medical Image Analysis: A Survey](https://mlsanity.com/paper/2411.05824/) (2024)
- [Dataset Distillation-based Hybrid Federated Learning on Non-IID Data](https://mlsanity.com/paper/2409.17517/) (2024)
- [Research in Collaborative Learning Does Not Serve Cross-Silo Federated Learning in Practice](https://mlsanity.com/paper/2510.12595/) (2025)
- [Exploring Federated Self-Supervised Learning for General Purpose Audio Understanding](https://mlsanity.com/paper/2402.02889/) (2024)
- [FedAnchor: Enhancing Federated Semi-Supervised Learning with Label Contrastive Loss for Unlabeled Clients](https://mlsanity.com/paper/2402.10191/) (2024)
- [FedCTTA: A Collaborative Approach to Continual Test-Time Adaptation in Federated Learning](https://mlsanity.com/paper/2505.13643/) (2025)
- [Distributionally Robust Alignment for Medical Federated Vision-Language Pre-training Under Data Heterogeneity](https://mlsanity.com/paper/2404.03854/) (2024)
- [Federated Self-Supervised Learning for One-Shot Cross-Modal and Cross-Imaging Technique Segmentation](https://mlsanity.com/paper/2503.23507/) (2025)
- [FSSUAVL: A Discriminative Framework using Vision Models for Federated Self-Supervised Audio and Image Understanding](https://mlsanity.com/paper/2504.09516/) (2025)
- [IPBA: Imperceptible Perturbation Backdoor Attack in Federated Self-Supervised Learning](https://mlsanity.com/paper/2508.08031/) (2025)
- [Foundational Models and Federated Learning: Survey, Taxonomy, Challenges and Practical Insights](https://mlsanity.com/paper/2509.05142/) (2025)
- [CATCHFed: Efficient Unlabeled Data Utilization for Semi-Supervised Federated Learning in Limited Labels Environments](https://mlsanity.com/paper/2511.11778/) (2025)
- [Understanding the Robustness of Distributed Self-Supervised Learning Frameworks Against Non-IID Data](https://mlsanity.com/paper/2607.02447/) (2026)
- [FedShift: Robust Federated Learning Aggregation Scheme in Resource Constrained Environment via Weight Shifting](https://mlsanity.com/paper/2402.01070/) (2024)
- [FedRepOpt: Gradient Re-parametrized Optimizers in Federated Learning](https://mlsanity.com/paper/2409.15898/) (2024)
- [Soft Separation and Distillation: Toward Global Uniformity in Federated Unsupervised Learning](https://mlsanity.com/paper/2508.01251/) (2025)
- [Cooperative Pseudo Labeling for Unsupervised Federated Classification](https://mlsanity.com/paper/2510.10100/) (2025)
- [Towards Heterogeneous Quantum Federated Learning: Challenges and Solutions](https://mlsanity.com/paper/2511.22148/) (2025)
- [ADCA: Attention-Driven Multi-Party Collusion Attack in Federated Self-Supervised Learning](https://mlsanity.com/paper/2602.05612/) (2026)
- [Federated Active Learning Under Extreme Non-IID and Global Class Imbalance](https://mlsanity.com/paper/2603.10341/) (2026)
- [Flower Hub: A Reproducible Benchmarking Platform for Federated Learning in Simulation and Deployment](https://mlsanity.com/paper/2608.25114/) (2026)
- [FedDRAW: Federated Dual Reputation Annealing Weighting for Heterogeneous Multi-Institutional Chest Radiograph Classification](https://mlsanity.com/paper/2609.05223/) (2026)

## Abstract (arXiv)

The ubiquity of camera-enabled devices has led to large amounts of unlabeled image data being produced at the edge. The integration of self-supervised learning (SSL) and federated learning (FL) into one coherent system can potentially offer data privacy guarantees while also advancing the quality and robustness of the learned visual representations without needing to move data around. However, client bias and divergence during FL aggregation caused by data heterogeneity limits the performance of learned visual representations on downstream tasks. In this paper, we propose a new aggregation strategy termed Layer-wise Divergence Aware Weight Aggregation (L-DAWA) to mitigate the influence of client bias and divergence during FL aggregation. The proposed method aggregates weights at the layer-level according to the measure of angular divergence between the clients' model and the global model. Extensive experiments with cross-silo and cross-device settings on CIFAR-10/100 and Tiny ImageNet datasets demonstrate that our methods are effective and obtain new SOTA performance on both contrastive and non-contrastive SSL approaches.