mlsanityData to 7 Oct 2026

Federated Learning with Position-Aware Neurons

arXiv 2203.14666 ↗First posted 28 Mar 2022cs.CV cs.DC cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) fuses collaborative models from local nodes without centralizing users' data. The permutation invariance property of neural networks and the non-i.i.d. data across clients make the locally updated parameters imprecisely aligned, disabling the coordinate-based parameter averaging. Traditional neurons do not explicitly consider position information. Hence, we propose Position-Aware Neurons (PANs) as an alternative, fusing position-related values (i.e., position encodings) into neuron outputs. PANs couple themselves to their positions and minimize the possibility of dislocation, even updating on heterogeneous data. We turn on/off PANs to disable/enable the permutation invariance property of neural networks. PANs are tightly coupled with positions when applied to FL, making parameters across clients pre-aligned and facilitating coordinate-based parameter averaging. PANs are algorithm-agnostic and could universally improve existing FL algorithms. Furthermore, "FL with PANs" is simple to implement and computationally friendly.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 31 of its 55 references that are in the map. First posted on arXiv 28 Mar 2022.

Builds on

31 of 55 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  7. 2019Advances and Open Problems in Federated Learningcited by 3,021
  8. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  9. 2018Group Normalizationcited by 1,514
  10. 2018Self-Attention with Relative Position Representationscited by 1,112
  11. 2018Federated Learning with Non-IID Datacited by 1,099
  12. 2020Adaptive Federated Optimizationcited by 1,036
19 more
  1. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  2. 2018LEAF: A Benchmark for Federated Settingscited by 780
  3. 2021Model-Contrastive Federated Learningcited by 762
  4. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  5. 2020Federated Learning with Matched Averagingcited by 564
  6. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  7. 2017Convolutional Sequence to Sequence Learningcited by 550
  8. 2019Federated Machine Learning: Concept and Applicationscited by 409
  9. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  10. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
  11. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
  12. 2019Model Fusion via Optimal Transportcited by 217
  13. 2017EMNIST: an extension of MNIST to handwritten letterscited by 212
  14. 2018Large scale distributed neural network training through online distillationcited by 144
  15. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 135
  16. 2020Positional Encoding as Spatial Inductive Bias in GANscited by 69
  17. 2020What Do Position Embeddings Learn? An Empirical Study of Pre-Trained Language Model Positional Encodingcited by 52
  18. 2021Fed2: Feature-Aligned Federated Learningcited by 42
  19. 2019Statistical Model Aggregation via Parameter Matchingcited by 17

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 7; 2024: 10; 2025: 3.

24 papers.

Sort:

  1. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
  2. 2024Towards Efficient Replay in Federated Incremental Learningcited by 33
  3. 2023Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedupcited by 23
  4. 2023A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiencycited by 21
  5. 2022Wasserstein Barycenter-based Model Fusion and Linear Mode Connectivity of Neural Networkscited by 14
  6. 2024AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Modelscited by 11
  7. 2023Federated Learning with Manifold Regularization and Normalized Update Reaggregationcited by 8
  8. 2024FedUV: Uniformity and Variance for Heterogeneous Federated Learningcited by 8
  9. 2023FedFN: Feature Normalization for Alleviating Data Heterogeneity Problem in Federated Learningcited by 7
  10. 2024Improving Model Fusion by Training-time Neuron Alignment with Fixed Neuron Anchorscited by 7
  11. 2024MAP: Model Aggregation and Personalization in Federated Learning with Incomplete Classescited by 7
  12. 2022Avoid Overfitting User Specific Information in Federated Keyword Spottingcited by 4
12 more
  1. 2024Federated Learning Client Pruning for Noisy Labelscited by 4
  2. 2023MrTF: Model Refinery for Transductive Federated Learningcited by 2
  3. 2024Weight Scope Alignment: A Frustratingly Easy Method for Model Mergingcited by 2
  4. 2025FedGA: A Fair Federated Learning Framework Based on the Gini Coefficientcited by 2
  5. 2022Tensor Decomposition based Personalized Federated Learningcited by 1
  6. 2023Personalized Federated Learning via Gradient Modulation for Heterogeneous Text Summarizationcited by 1
  7. 2024Exploring and Exploiting the Asymmetric Valley of Deep Neural Networkscited by 1
  8. 2024Visualizing, Rethinking, and Mining the Loss Landscape of Deep Neural Networkscited by 1
  9. 2023FedAgg: Adaptive Federated Learning with Aggregated Gradientscited by 0
  10. 2024Selective Knowledge Sharing for Personalized Federated Learning Under Capacity Heterogeneitycited by 0
  11. 2025FedDyMem: Efficient Federated Learning with Dynamic Memory and Memory-Reduce for Unsupervised Image Anomaly Detectioncited by 0
  12. 2025UniVarFL: Uniformity and Variance Regularized Federated Learning for Heterogeneous Datacited by 0