# DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learning

arXiv: https://arxiv.org/abs/2111.00465 · HTML: https://mlsanity.com/paper/2111.00465/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 34 references that are in the map.
- First posted on arXiv 31 Oct 2021.

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Flower: A Friendly Federated Learning Research Framework](https://mlsanity.com/paper/2007.14390/) (2020)
- [Deep Learning with Limited Numerical Precision](https://mlsanity.com/paper/1502.02551/) (2015)
- [FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization](https://mlsanity.com/paper/1909.13014/) (2019)
- [Model Pruning Enables Efficient Federated Learning on Edge Devices](https://mlsanity.com/paper/1909.12326/) (2019)
- [FetchSGD: Communication-Efficient Federated Learning with Sketching](https://mlsanity.com/paper/2007.07682/) (2020)
- [Training Deep Neural Networks with 8-bit Floating Point Numbers](https://mlsanity.com/paper/1812.08011/) (2018)
- [Scalable Methods for 8-bit Training of Neural Networks](https://mlsanity.com/paper/1805.11046/) (2018)
- [UVeQFed: Universal Vector Quantization for Federated Learning](https://mlsanity.com/paper/2006.03262/) (2020)
- [Training Quantized Nets: A Deeper Understanding](https://mlsanity.com/paper/1706.02379/) (2017)
- [Stochastic Distributed Learning with Gradient Quantization and Variance Reduction](https://mlsanity.com/paper/1904.05115/) (2019)
- [Adaptive Quantization of Model Updates for Communication-Efficient Federated Learning](https://mlsanity.com/paper/2102.04487/) (2021)
- [Fast-Convergent Federated Learning](https://mlsanity.com/paper/2007.13137/) (2020)
- [Federated Learning With Quantized Global Model Updates](https://mlsanity.com/paper/2006.10672/) (2020)
- [Can Federated Learning Save The Planet?](https://mlsanity.com/paper/2010.06537/) (2020)
- [Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning](https://mlsanity.com/paper/2011.04050/) (2020)
- [FEDZIP: A Compression Framework for Communication-Efficient Federated Learning](https://mlsanity.com/paper/2102.01593/) (2021)
- [Fractional Skipping: Towards Finer-Grained Dynamic CNN Inference](https://mlsanity.com/paper/2001.00705/) (2020)
- [FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training](https://mlsanity.com/paper/2012.13113/) (2020)
- [DeepMood: Modeling Mobile Phone Typing Dynamics for Mood Detection](https://mlsanity.com/paper/1803.08986/) (2018)
- [A Smart Home Gateway Platform for Data Collection and Awareness](https://mlsanity.com/paper/1804.01242/) (2018)

## Built on by (most cited first, 30 of 41)

- [Heterogeneous Federated Learning: State-of-the-art and Research Challenges](https://mlsanity.com/paper/2307.10616/) (2023)
- [Adaptive Compression for Communication-Efficient Distributed Training](https://mlsanity.com/paper/2211.00188/) (2022)
- [Distributed Deep Reinforcement Learning Based Gradient Quantization for Federated Learning Enabled Vehicle Edge Computing](https://mlsanity.com/paper/2407.08462/) (2024)
- [Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization](https://mlsanity.com/paper/2405.15861/) (2024)
- [Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devices](https://mlsanity.com/paper/2311.18129/) (2023)
- [DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation](https://mlsanity.com/paper/2405.06368/) (2024)
- [FedBAT: Communication-Efficient Federated Learning via Learnable Binarization](https://mlsanity.com/paper/2408.03215/) (2024)
- [Distributed Linear Bandits under Communication Constraints](https://mlsanity.com/paper/2211.02212/) (2022)
- [A Communication-Efficient Adaptive Algorithm for Federated Learning under Cumulative Regret](https://mlsanity.com/paper/2301.08869/) (2023)
- [AQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection Strategy](https://mlsanity.com/paper/2308.00258/) (2023)
- [Dynamic Stashing Quantization for Efficient Transformer Training](https://mlsanity.com/paper/2303.05295/) (2023)
- [FedDM: Enhancing Communication Efficiency and Handling Data Heterogeneity in Federated Diffusion Models](https://mlsanity.com/paper/2407.14730/) (2024)
- [The Panaceas for Improving Low-Rank Decomposition in Communication-Efficient Federated Learning](https://mlsanity.com/paper/2505.23176/) (2025)
- [ResFed: Communication Efficient Federated Learning by Transmitting Deep Compressed Residuals](https://mlsanity.com/paper/2212.05602/) (2022)
- [Masked Random Noise for Communication Efficient Federated Learning](https://mlsanity.com/paper/2408.03220/) (2024)
- [FedRTS: Federated Robust Pruning via Combinatorial Thompson Sampling](https://mlsanity.com/paper/2501.19122/) (2025)
- [Network Adaptive Federated Learning: Congestion and Lossy Compression](https://mlsanity.com/paper/2301.04430/) (2023)
- [Federated Learning: A Cutting-Edge Survey of the Latest Advancements and Applications](https://mlsanity.com/paper/2310.05269/) (2023)
- [FedCode: Communication-Efficient Federated Learning via Transferring Codebooks](https://mlsanity.com/paper/2311.09270/) (2023)
- [Communication-Efficient Federated Knowledge Graph Embedding with Entity-Wise Top-K Sparsification](https://mlsanity.com/paper/2406.13225/) (2024)
- [Converge Faster, Talk Less: Hessian-Informed Federated Zeroth-Order Optimization](https://mlsanity.com/paper/2506.02370/) (2025)
- [SignSGD with Federated Voting](https://mlsanity.com/paper/2403.16372/) (2024)
- [Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networks](https://mlsanity.com/paper/2501.11267/) (2025)
- [Communication-Efficient {Federated} Learning Using Censored Heavy Ball Descent](https://mlsanity.com/paper/2209.11944/) (2022)
- [Towards Understanding Adversarial Transferability in Federated Learning](https://mlsanity.com/paper/2310.00616/) (2023)
- [Fed-CVLC: Compressing Federated Learning Communications with Variable-Length Codes](https://mlsanity.com/paper/2402.03770/) (2024)
- [Energy-Efficient Wireless Federated Learning via Doubly Adaptive Quantization](https://mlsanity.com/paper/2402.12957/) (2024)
- [Privacy-Aware Randomized Quantization via Linear Programming](https://mlsanity.com/paper/2406.02599/) (2024)
- [Towards Federated Learning with On-device Training and Communication in 8-bit Floating Point](https://mlsanity.com/paper/2407.02610/) (2024)
- [Personalized Federated Learning for Generative AI-Assisted Semantic Communications](https://mlsanity.com/paper/2410.02450/) (2024)

## Abstract (arXiv)

Federated Learning (FL) is a powerful technique for training a model on a server with data from several clients in a privacy-preserving manner. In FL, a server sends the model to every client, who then train the model locally and send it back to the server. The server aggregates the updated models and repeats the process for several rounds. FL incurs significant communication costs, in particular when transmitting the updated local models from the clients back to the server. Recently proposed algorithms quantize the model parameters to efficiently compress FL communication. These algorithms typically have a quantization level that controls the compression factor. We find that dynamic adaptations of the quantization level can boost compression without sacrificing model quality. First, we introduce a time-adaptive quantization algorithm that increases the quantization level as training progresses. Second, we introduce a client-adaptive quantization algorithm that assigns each individual client the optimal quantization level at every round. Finally, we combine both algorithms into DAdaQuant, the doubly-adaptive quantization algorithm. Our experiments show that DAdaQuant consistently improves clientrightarrowserver compression, outperforming the strongest non-adaptive baselines by up to 2.8×.