mlsanityData to 7 Oct 2026

EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning

arXiv 2108.08842 ↗First posted 19 Aug 2021cs.LG cs.AI cs.DScited by 53

Not on the map. Why?

Abstract (arXiv)

Distributed Mean Estimation (DME) is a central building block in federated learning, where clients send local gradients to a parameter server for averaging and updating the model. Due to communication constraints, clients often use lossy compression techniques to compress the gradients, resulting in estimation inaccuracies. DME is more challenging when clients have diverse network conditions, such as constrained communication budgets and packet losses. In such settings, DME techniques often incur a significant increase in the estimation error leading to degraded learning performance. In this work, we propose a robust DME technique named EDEN that naturally handles heterogeneous communication budgets and packet losses. We derive appealing theoretical guarantees for EDEN and evaluate it empirically. Our results demonstrate that EDEN consistently improves over state-of-the-art DME techniques.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 36 of its 82 references that are in the map. First posted on arXiv 19 Aug 2021.

Builds on

36 of 82 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  7. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  8. 2020Adaptive Federated Optimizationcited by 1,036
  9. 2018LEAF: A Benchmark for Federated Settingscited by 780
  10. 2014Convex Optimization: Algorithms and Complexitycited by 569
  11. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  12. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 471
24 more
  1. 2018Sparsified SGD with Memorycited by 373
  2. 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 364
  3. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
  4. 2019Robust Aggregation for Federated Learningcited by 314
  5. 2021A Field Guide to Federated Optimizationcited by 312
  6. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 306
  7. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 253
  8. 2018The Convergence of Sparsified Gradient Methodscited by 228
  9. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 162
  10. 2020On Biased Compression for Distributed Learningcited by 140
  11. 2016Distributed Mean Estimation with Limited Communicationcited by 134
  12. 2021EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedbackcited by 131
  13. 2015Practical and Optimal LSH for Angular Distancecited by 105
  14. 2016Orthogonal Random Featurescited by 89
  15. 2019Communication-efficient distributed SGD with Sketchingcited by 81
  16. 2021MARINA: Faster Non-Convex Distributed Learning with Compressioncited by 77
  17. 2020From Local SGD to Local Fixed-Point Methods for Federated Learningcited by 74
  18. 2019Scaling Distributed Machine Learning with In-Network Aggregationcited by 62
  19. 2021DRIVE: One-bit Distributed Mean Estimationcited by 52
  20. 2018Structured Evolution with Compact Architectures for Scalable Policy Optimizationcited by 48
  21. 2020Uncertainty Principle for Communication Compression in Distributed and Federated Learning and the Search for an Optimal Compressorcited by 45
  22. 2017The Unreasonable Effectiveness of Structured Random Orthogonal Embeddingscited by 43
  23. 2016Randomized Distributed Mean Estimation: Accuracy vs Communicationcited by 39
  24. 2020How to send a real number using a single bit (and some shared randomness)cited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 4; 2023: 11; 2024: 7; 2025: 8; 2026: 22.

53 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  2. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 116
  3. 2025Quartet: Native FP4 Training Can Be Optimal for Large Language Modelscited by 37
  4. 2022Sparse Random Networks for Communication-Efficient Federated Learningcited by 32
  5. 2023DoCoFL: Downlink Compression for Cross-Device Federated Learningcited by 27
  6. 2023Matrix Compression via Randomized Low Rank and Low Precision Factorizationcited by 25
  7. 2025QuEST: Stable Training of LLMs with 1-Bit Weights and Activationscited by 21
  8. 2024Pushing the Limits of Large Language Model Quantization via the Linearity Theoremcited by 20
  9. 2026Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimationcited by 16
  10. 2023THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compressioncited by 15
  11. 2023Fast Optimal Locally Private Mean Estimation via Random Projectionscited by 14
  12. 2023Exact Optimality of Communication-Privacy-Utility Tradeoffs in Distributed Mean Estimationcited by 9
38 more
  1. 2022ScionFL: Efficient and Robust Secure Quantized Aggregationcited by 8
  2. 2021SDR: Efficient Neural Re-ranking using Succinct Document Representationcited by 7
  3. 2022QUIC-FL: Quick Unbiased Compression for Federated Learningcited by 7
  4. 2022GlueFL: Reconciling Client Sampling and Model Masking for Bandwidth Efficient Federated Learningcited by 7
  5. 2026A Note on TurboQuant and the Earlier DRIVE/EDEN Line of Workcited by 7
  6. 2023Adaptive Compression in Federated Learning via Side Informationcited by 6
  7. 2023LASER: Linear Compression in Wireless Distributed Optimizationcited by 6
  8. 2024Beyond Throughput and Compression Ratios: Towards High End-to-end Utility of Gradient Compressioncited by 6
  9. 2026High-Rate Quantized Matrix Multiplication Icited by 6
  10. 2026Provable Quantization with Randomized Hadamard Transformcited by 6
  11. 2024Blockchain-empowered Federated Learning: Benefits, Challenges, and Solutionscited by 5
  12. 2025Sequential 1-bit Mean Estimation with Near-Optimal Sample Complexitycited by 5
  13. 2026Order-Optimal Sequential 1-Bit Mean Estimation in General Tail Regimescited by 5
  14. 2024Masked Random Noise for Communication Efficient Federated Learningcited by 4
  15. 2026DynamiQ: Accelerating Gradient Synchronization using Compressed Multi-hop All-reducecited by 4
  16. 2026Quantizing With Randomized Hadamard Transforms: Efficient Heuristic Now Provencited by 4
  17. 2023Correlation Aware Sparsified Mean Estimation Using Random Projectioncited by 3
  18. 2023FedCode: Communication-Efficient Federated Learning via Transferring Codebookscited by 3
  19. 2024EncCluster: Scalable Functional Encryption in Federated Learning through Weight Clustering and Probabilistic Filterscited by 3
  20. 2026Revisiting RaBitQ and TurboQuant: A Symmetric Comparison of Methods, Theory, and Experimentscited by 3
  21. 2023Federated Fine-Tuning of Foundation Models via Probabilistic Maskingcited by 2
  22. 2024Optimal and Near-Optimal Adaptive Vector Quantizationcited by 2
  23. 2025FedFetch: Faster Federated Learning with Adaptive Downstream Prefetchingcited by 2
  24. 2026Approximating Uniform Random Rotations by Two-Block Structured Hadamard Rotations in High Dimensionscited by 2
  25. 2024Noise-Robust and Resource-Efficient ADMM-based Federated Learningcited by 1
  26. 2025PREAMBLE: Private and Efficient Aggregation via Block Sparse Vectorscited by 1
  27. 2025Personalized Federated Learning with Bidirectional Communication Compression via One-Bit Random Sketchingcited by 1
  28. 2025CafeQ: Calibration-free Quantization via Learned Transformations and Adaptive Roundingcited by 1
  29. 2026Contraction-Gauge Preconditioning for Quantized Matrix Multiplicationcited by 1
  30. 2026Entropy-Constrained Adaptive Stochastic Quantizationcited by 1
  31. 2026Block-Sphere Vector Quantizationcited by 0
  32. 2026SSTQ:Privacy-Preserving Vector Quantization via Subsampled Stochastic TurboQuantcited by 0
  33. 2026VQ-bench: A Composable Vector Quantization Frameworkcited by 0
  34. 2026RSLM: Training-Free Vector Quantization for Approximate Nearest Neighbor Searchcited by 0
  35. 2026Communication-Efficient Personalized Federated Learning via Layer-Wise Multi-Threshold Random Sketchingcited by 0
  36. 2026A Note on Scaling in Randomly Rotated Quantization and Its Connection to the CDEF +1 Pythagorean Relationcited by 0
  37. 2026KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocationcited by 0
  38. 2026TORQUE: Optimizing What (not) to Quantize Before and After Rotationcited by 0

50 most cited of 53 All 53 (JSON)