mlsanityData to 7 Oct 2026

Fed2: Feature-Aligned Federated Learning

arXiv 2111.14248 ↗First posted 28 Nov 2021cs.LG cs.AIcited by 42

Not on the map. Why?

Abstract (arXiv)

Federated learning learns from scattered data by fusing collaborative models from local nodes. However, the conventional coordinate-based model averaging by FedAvg ignored the random information encoded per parameter and may suffer from structural feature misalignment. In this work, we propose Fed2, a feature-aligned federated learning framework to resolve this issue by establishing a firm structure-feature alignment across the collaborative models. Fed2 is composed of two major designs: First, we design a feature-oriented model structure adaptation method to ensure explicit feature allocation in different neural network structures. Applying the structure adaptation to collaborative models, matchable structures with similar feature information can be initialized at the very early training stage. During the federated learning process, we then propose a feature paired averaging scheme to guarantee aligned feature distribution and maintain no feature fusion conflicts under either IID or non-IID scenarios. Eventually, Fed2 could effectively enhance the federated learning convergence performance under extensive homo- and heterogeneous settings, providing excellent convergence speed, accuracy, and computation/communication efficiency.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 18 of its 24 references that are in the map. First posted on arXiv 28 Nov 2021.

Builds on

18 of 24 references.

References · 18

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  6. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  7. 2018Group Normalizationcited by 1,514
  8. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  9. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  10. 2018Federated Learning with Non-IID Datacited by 1,099
  11. 2020Federated Learning with Matched Averagingcited by 564
  12. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
6 more
  1. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  2. 2015Understanding Neural Networks Through Deep Visualizationcited by 332
  3. 2019Model Fusion via Optimal Transportcited by 217
  4. 2016Do semantic parts emerge in Convolutional Neural Networks?cited by 23
  5. 2019Robust Federated Learning Through Representation Matching and Adaptive Hyper-parameterscited by 16
  6. 2018Distilling Critical Paths in Convolutional Neural Networkscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 15; 2024: 10; 2025: 5; 2026: 3.

42 papers.

Sort:

By area:

  1. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
  2. 2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
  3. 2022Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learningcited by 52
  4. 2022FedFA: Federated Learning with Feature Anchors to Align Features and Classifiers for Heterogeneous Datacited by 49
  5. 2022pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learningcited by 44
  6. 2023Recent Advances on Federated Learning: A Systematic Surveycited by 40
  7. 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
  8. 2023pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuningcited by 38
  9. 2022TCT: Convexifying Federated Learning using Bootstrapped Neural Tangent Kernelscited by 26
  10. 2022Federated Learning with Position-Aware Neuronscited by 24
  11. 2022FRAug: Tackling Federated Learning with Non-IID Features via Representation Augmentationcited by 24
  12. 2022FedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networkscited by 23
30 more
  1. 2023Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedupcited by 23
  2. 2022Mitigating Data Heterogeneity in Federated Learning with Data Augmentationcited by 21
  3. 2024pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learningcited by 21
  4. 2024Federated Model Heterogeneous Matryoshka Representation Learningcited by 21
  5. 2023FedHCA2: Towards Hetero-Client Federated Multi-Task Learningcited by 17
  6. 2022Wasserstein Barycenter-based Model Fusion and Linear Mode Connectivity of Neural Networkscited by 14
  7. 2025Can Textual Gradient Work in Federated Learning?cited by 14
  8. 2024AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Modelscited by 11
  9. 2023pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharingcited by 9
  10. 2023Making Batch Normalization Great in Federated Deep Learningcited by 8
  11. 2023Federated Learning with Manifold Regularization and Normalized Update Reaggregationcited by 8
  12. 2023FedFN: Feature Normalization for Alleviating Data Heterogeneity Problem in Federated Learningcited by 7
  13. 2025Fed-HeLLo: Efficient Federated Foundation Model Fine-Tuning with Heterogeneous LoRA Allocationcited by 7
  14. 2023DC-CCL: Device-Cloud Collaborative Controlled Learning for Large Vision Modelscited by 6
  15. 2024Improving Local Training in Federated Learning via Temperature Scalingcited by 5
  16. 2024Federated brain tumor segmentation: an extensive benchmarkcited by 5
  17. 2023Towards Open Federated Learning Platforms: Survey and Vision from Technical and Legal Perspectivescited by 3
  18. 2023FedBug: A Bottom-Up Gradual Unfreezing Framework for Federated Learningcited by 3
  19. 2024pFedAFM: Adaptive Feature Mixture for Batch-Level Personalization in Heterogeneous Federated Learningcited by 3
  20. 2024Towards Optimal Customized Architecture for Heterogeneous Federated Learning with Contrastive Cloud-Edge Model Decouplingcited by 2
  21. 2024Weight Scope Alignment: A Frustratingly Easy Method for Model Mergingcited by 2
  22. 2023Learning From Drift: Federated Learning on Non-IID Data via Drift Regularizationcited by 1
  23. 2024Exploring and Exploiting the Asymmetric Valley of Deep Neural Networkscited by 1
  24. 2024Federated Cross-Training Learners for Robust Generalization under Data Heterogeneitycited by 1
  25. 2025The Transition from Centralized Machine Learning to Federated Learning for Mental Health in Education: A Survey of Current Methods and Future Directionscited by 1
  26. 2025ATR-Bench: A Federated Learning Benchmark for Adaptation, Trust, and Reasoningcited by 0
  27. 2025FedLLM-Align: Feature Extraction From Heterogeneous Clientscited by 0
  28. 2026FedCova: Robust Federated Covariance Learning Against Noisy Labelscited by 0
  29. 2026AlignFed: Alignment-Aware Asynchronous Federated Fine-Tuning for Large Language Models in Heterogeneous Edge Environmentscited by 0
  30. 2026CRIP: Channel Level Representation Injection for Personalized One-Shot Federated Learningcited by 0