mlsanityData to 7 Oct 2026

DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learning

arXiv 2111.00465 ↗First posted 31 Oct 2021cs.LGcited by 40

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a powerful technique for training a model on a server with data from several clients in a privacy-preserving manner. In FL, a server sends the model to every client, who then train the model locally and send it back to the server. The server aggregates the updated models and repeats the process for several rounds. FL incurs significant communication costs, in particular when transmitting the updated local models from the clients back to the server. Recently proposed algorithms quantize the model parameters to efficiently compress FL communication. These algorithms typically have a quantization level that controls the compression factor. We find that dynamic adaptations of the quantization level can boost compression without sacrificing model quality. First, we introduce a time-adaptive quantization algorithm that increases the quantization level as training progresses. Second, we introduce a client-adaptive quantization algorithm that assigns each individual client the optimal quantization level at every round. Finally, we combine both algorithms into DAdaQuant, the doubly-adaptive quantization algorithm. Our experiments show that DAdaQuant consistently improves clientrightarrowserver compression, outperforming the strongest non-adaptive baselines by up to 2.8×.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 25 of its 34 references that are in the map. First posted on arXiv 31 Oct 2021.

Builds on

25 of 34 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2018LEAF: A Benchmark for Federated Settingscited by 780
  6. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
  7. 2015Deep Learning with Limited Numerical Precisioncited by 368
  8. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  9. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 219
  10. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 178
  11. 2018Training Deep Neural Networks with 8-bit Floating Point Numberscited by 163
  12. 2018Scalable Methods for 8-bit Training of Neural Networkscited by 139
13 more
  1. 2020UVeQFed: Universal Vector Quantization for Federated Learningcited by 103
  2. 2017Training Quantized Nets: A Deeper Understandingcited by 74
  3. 2019Stochastic Distributed Learning with Gradient Quantization and Variance Reductioncited by 64
  4. 2020Fast-Convergent Federated Learningcited by 61
  5. 2021Adaptive Quantization of Model Updates for Communication-Efficient Federated Learningcited by 60
  6. 2020Federated Learning With Quantized Global Model Updatescited by 54
  7. 2020Can Federated Learning Save The Planet?cited by 52
  8. 2020Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learningcited by 37
  9. 2021FEDZIP: A Compression Framework for Communication-Efficient Federated Learningcited by 27
  10. 2020Fractional Skipping: Towards Finer-Grained Dynamic CNN Inferencecited by 23
  11. 2020FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Trainingcited by 23
  12. 2018DeepMood: Modeling Mobile Phone Typing Dynamics for Mood Detectioncited by 10
  13. 2018A Smart Home Gateway Platform for Data Collection and Awarenesscited by 4

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 9; 2024: 16; 2025: 11.

40 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  2. 2024Distributed Deep Reinforcement Learning Based Gradient Quantization for Federated Learning Enabled Vehicle Edge Computingcited by 15
  3. 2022Adaptive Compression for Communication-Efficient Distributed Trainingcited by 14
  4. 2024Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimizationcited by 12
  5. 2023Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devicescited by 11
  6. 2024DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptationcited by 9
  7. 2024FedBAT: Communication-Efficient Federated Learning via Learnable Binarizationcited by 9
  8. 2022Distributed Linear Bandits under Communication Constraintscited by 8
  9. 2023A Communication-Efficient Adaptive Algorithm for Federated Learning under Cumulative Regretcited by 7
  10. 2023AQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection Strategycited by 7
  11. 2023Dynamic Stashing Quantization for Efficient Transformer Trainingcited by 6
  12. 2024FedDM: Enhancing Communication Efficiency and Handling Data Heterogeneity in Federated Diffusion Modelscited by 5
28 more
  1. 2025The Panaceas for Improving Low-Rank Decomposition in Communication-Efficient Federated Learningcited by 5
  2. 2022ResFed: Communication Efficient Federated Learning by Transmitting Deep Compressed Residualscited by 4
  3. 2024Masked Random Noise for Communication Efficient Federated Learningcited by 4
  4. 2025FedRTS: Federated Robust Pruning via Combinatorial Thompson Samplingcited by 4
  5. 2023Network Adaptive Federated Learning: Congestion and Lossy Compressioncited by 3
  6. 2023Federated Learning: A Cutting-Edge Survey of the Latest Advancements and Applicationscited by 3
  7. 2023FedCode: Communication-Efficient Federated Learning via Transferring Codebookscited by 3
  8. 2024Communication-Efficient Federated Knowledge Graph Embedding with Entity-Wise Top-K Sparsificationcited by 3
  9. 2025Converge Faster, Talk Less: Hessian-Informed Federated Zeroth-Order Optimizationcited by 3
  10. 2024SignSGD with Federated Votingcited by 2
  11. 2025Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networkscited by 2
  12. 2022Communication-Efficient {Federated} Learning Using Censored Heavy Ball Descentcited by 1
  13. 2023Towards Understanding Adversarial Transferability in Federated Learningcited by 1
  14. 2024Fed-CVLC: Compressing Federated Learning Communications with Variable-Length Codescited by 1
  15. 2024Privacy-Aware Randomized Quantization via Linear Programmingcited by 1
  16. 2024Towards Federated Learning with On-device Training and Communication in 8-bit Floating Pointcited by 1
  17. 2024Personalized Federated Learning for Generative AI-Assisted Semantic Communicationscited by 1
  18. 2024Adaptive Quantization Resolution and Power Control for Federated Learning over Cell-free Networkscited by 1
  19. 2024Accelerating Energy-Efficient Federated Learning in Cell-Free Networks with Adaptive Quantizationcited by 1
  20. 2025Characterizing the Accuracy-Communication-Privacy Trade-off in Distributed Stochastic Convex Optimizationcited by 1
  21. 2025FedBiF: Communication-Efficient Federated Learning via Bits Freezingcited by 1
  22. 2025Time-varying Mixing Matrix Design for Energy-efficient Decentralized Federated Learningcited by 1
  23. 2024FedShift: Robust Federated Learning Aggregation Scheme in Resource Constrained Environment via Weight Shiftingcited by 0
  24. 2024SADDLe: Sharpness-Aware Decentralized Deep Learning with Heterogeneous Datacited by 0
  25. 2025FedX: Adaptive Model Decomposition and Quantization for IoT Federated Learningcited by 0
  26. 2025Sparsification Under Siege: Dual-Level Defense Against Poisoning in Communication-Efficient Federated Learningcited by 0
  27. 2025Robust Federated Learning against Model Perturbation in Edge Networkscited by 0
  28. 2025Enhancing Communication Efficiency in FL with Adaptive Gradient Quantization and Communication Frequency Optimizationcited by 0