mlsanityData to 14 Oct 2026

FedWSQ: Efficient Federated Learning with Weight Standardization and Distribution-Aware Non-Uniform Quantization

arXiv 2506.23516 ↗First posted 30 Jun 2025cs.LG cs.AI cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) often suffers from performance degradation due to key challenges such as data heterogeneity and communication constraints. To address these limitations, we present a novel FL framework called FedWSQ, which integrates weight standardization (WS) and the proposed distribution-aware non-uniform quantization (DANUQ). WS enhances FL performance by filtering out biased components in local updates during training, thereby improving the robustness of the model against data heterogeneity and unstable client participation. In addition, DANUQ minimizes quantization errors by leveraging the statistical properties of local model updates. As a result, FedWSQ significantly reduces communication overhead while maintaining superior model accuracy. Extensive experiments on FL benchmark datasets demonstrate that FedWSQ consistently outperforms existing FL methods across various challenging FL settings, including extreme data heterogeneity and ultra-low-bit communication scenarios.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 46 of its 57 references that are in the map. First posted on arXiv 30 Jun 2025.

Builds on

46 of 57 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  7. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  8. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  9. 2018Group Normalizationcited by 1,524
  10. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  11. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  12. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
34 more
  1. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
  2. 2020Adaptive Federated Optimizationcited by 1,046
  3. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  4. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 642
  5. 2019Learned Step Size Quantizationcited by 628
  6. 2018Local SGD Converges Fast and Communicates Littlecited by 605
  7. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  8. 2021A White Paper on Neural Network Quantizationcited by 472
  9. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  10. 20218-bit Optimizers via Block-wise Quantizationcited by 403
  11. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
  12. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 305
  13. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 278
  14. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 274
  15. 2016Trained Ternary Quantizationcited by 208
  16. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
  17. 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 181
  18. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 175
  19. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 160
  20. 2015Fixed Point Quantization of Deep Convolutional Networkscited by 143
  21. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 109
  22. 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
  23. 2019First Analysis of Local GD on Heterogeneous Datacited by 81
  24. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  25. 2025Optimizing Large Language Model Training Using FP4 Quantizationcited by 56
  26. 2022Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learningcited by 46
  27. 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
  28. 2022Multi-Level Branched Regularization for Federated Learningcited by 34
  29. 2024Relaxed Contrastive Learning for Federated Learningcited by 32
  30. 2022Communication-Efficient Federated Learning with Accelerated Client Gradientcited by 24
  31. 2022QReg: On Regularization Effects of Quantizationcited by 11
  32. 2023FedWon: Triumphing Multi-domain Federated Learning Without Normalizationcited by 11
  33. 2024QGen: On the Ability to Generalize in Quantization Aware Trainingcited by 7
  34. 2024Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspectivecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Quantization in Federated Learning: Methods, Challenges and Future Directionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024FedFQ: Federated Learning with Fine-Grained Quantization
  2. 2025FedHQ: Hybrid Runtime Quantization for Federated Learning
  3. 2023Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devices
  4. 2025FedSparQ: Adaptive Sparse Quantization with Error Feedback for Robust & Efficient Federated Learning
  5. 2021FedDQ: Communication-Efficient Federated Learning with Descending Quantization
  6. 2024FedShift: Robust Federated Learning Aggregation Scheme in Resource Constrained Environment via Weight Shifting
  7. 2023Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity
  8. 2024Communication-Efficient Federated Learning via Clipped Uniform Quantization