mlsanityData to 14 Oct 2026

DRIVE: One-bit Distributed Mean Estimation

arXiv 2105.08339 ↗First posted 18 May 2021cs.LG cs.DScited by 52

Not on the map. Why?

Abstract (arXiv)

We consider the problem where n clients transmit d-dimensional real-valued vectors using d(1+o(1)) bits each, in a manner that allows the receiver to approximately reconstruct their mean. Such compression problems naturally arise in distributed and federated learning. We provide novel mathematical results and derive computationally efficient algorithms that are more accurate than previous compression techniques. We evaluate our methods on a collection of distributed and federated learning tasks, using a variety of datasets, and show a consistent improvement over the state of the art.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). Builds on 23 of its 88 references that are in the map. First posted on arXiv 18 May 2021.

Builds on

23 of 88 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  6. 2020Adaptive Federated Optimizationcited by 1,046
  7. 2018LEAF: A Benchmark for Federated Settingscited by 792
  8. 2018Sparsified SGD with Memorycited by 376
  9. 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 370
  10. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 310
  11. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 256
  12. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 238
11 more
  1. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 179
  2. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 163
  3. 2016QSGD: Communication-Efficient SGD via Gradient Quantization and Encodingcited by 147
  4. 2020On Biased Compression for Distributed Learningcited by 140
  5. 2016Distributed Mean Estimation with Limited Communicationcited by 136
  6. 2018ATOMO: Communication-efficient Learning via Atomic Sparsificationcited by 136
  7. 2019Communication-efficient distributed SGD with Sketchingcited by 81
  8. 2020Uncertainty Principle for Communication Compression in Distributed and Federated Learning and the Search for an Optimal Compressorcited by 45
  9. 2016Randomized Distributed Mean Estimation: Accuracy vs Communicationcited by 39
  10. 2017Fast Exact k-Means, k-Medians and Bregman Divergence Clustering in 1Dcited by 15
  11. 2020How to send a real number using a single bit (and some shared randomness)cited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 7; 2023: 11; 2024: 10; 2025: 4; 2026: 18.

52 papers.

Sort:

By area:

  1. 2021EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learningcited by 55
  2. 2025Quartet: Native FP4 Training Can Be Optimal for Large Language Modelscited by 39
  3. 2022Sparse Random Networks for Communication-Efficient Federated Learningcited by 32
  4. 2023DoCoFL: Downlink Compression for Cross-Device Federated Learningcited by 28
  5. 2022Scaling Language Model Size in Cross-Device Federated Learningcited by 25
  6. 2023Matrix Compression via Randomized Low Rank and Low Precision Factorizationcited by 25
  7. 2025QuEST: Stable Training of LLMs with 1-Bit Weights and Activationscited by 22
  8. 2024Pushing the Limits of Large Language Model Quantization via the Linearity Theoremcited by 21
  9. 2026Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimationcited by 21
  10. 2023z-SignFedAvg: A Unified Stochastic Sign-based Compression for Federated Learningcited by 17
  11. 2023THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compressioncited by 15
  12. 2022Optimizing the Communication-Accuracy Trade-off in Federated Learning with Rate-Distortion Theorycited by 14
38 more
  1. 2022Correlated quantization for distributed mean estimation and optimizationcited by 14
  2. 2023Fast Optimal Locally Private Mean Estimation via Random Projectionscited by 14
  3. 2022High Dimensional Statistical Estimation under Uniformly Dithered One-bit Quantizationcited by 13
  4. 2023Exact Optimality of Communication-Privacy-Utility Tradeoffs in Distributed Mean Estimationcited by 9
  5. 2022ScionFL: Efficient and Robust Secure Quantized Aggregationcited by 8
  6. 2026A Note on TurboQuant and the Earlier DRIVE/EDEN Line of Workcited by 8
  7. 2026Provable Quantization with Randomized Hadamard Transformcited by 8
  8. 2021SDR: Efficient Neural Re-ranking using Succinct Document Representationcited by 7
  9. 2022QUIC-FL: Quick Unbiased Compression for Federated Learningcited by 7
  10. 2023Adaptive Compression in Federated Learning via Side Informationcited by 6
  11. 2023LASER: Linear Compression in Wireless Distributed Optimizationcited by 6
  12. 2024Beyond Throughput and Compression Ratios: Towards High End-to-end Utility of Gradient Compressioncited by 6
  13. 2026Order-Optimal Sequential 1-Bit Mean Estimation in General Tail Regimescited by 6
  14. 2026Quantizing With Randomized Hadamard Transforms: Efficient Heuristic Now Provencited by 6
  15. 2025Sequential 1-bit Mean Estimation with Near-Optimal Sample Complexitycited by 5
  16. 2024Improved Communication-Privacy Trade-offs in L2 Mean Estimation under Streaming Differential Privacycited by 4
  17. 2024Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspectivecited by 4
  18. 2024Masked Random Noise for Communication Efficient Federated Learningcited by 4
  19. 2026DynamiQ: Accelerating Gradient Synchronization using Compressed Multi-hop All-reducecited by 4
  20. 2023Correlation Aware Sparsified Mean Estimation Using Random Projectioncited by 3
  21. 2024Correlated Quantization for Faster Nonconvex Distributed Optimizationcited by 3
  22. 2024Learning with User-Level Local Differential Privacycited by 3
  23. 2026Revisiting RaBitQ and TurboQuant: A Symmetric Comparison of Methods, Theory, and Experimentscited by 3
  24. 2023Federated Fine-Tuning of Foundation Models via Probabilistic Maskingcited by 2
  25. 2024Optimal and Near-Optimal Adaptive Vector Quantizationcited by 2
  26. 2026Approximating Uniform Random Rotations by Two-Stage Structured Hadamard Rotations in High Dimensionscited by 2
  27. 2026Interaction Is Not Necessary for Order-Optimal 1-Bit Mean Estimationcited by 2
  28. 2023CORE: Common Random Reconstruction for Distributed Optimization with Provable Low Communication Complexitycited by 1
  29. 2024Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysiscited by 1
  30. 2024PAPAYA Federated Analytics Stack: Engineering Privacy, Scalability and Practicalitycited by 1
  31. 2025PREAMBLE: Private and Efficient Aggregation via Block Sparse Vectorscited by 1
  32. 2026Collaborative Compressors in Distributed Mean Estimation with Limited Communication Budgetcited by 1
  33. 2026Lynx: Progressive Speculative Quantization for accelerating KV Transfer in Long-Context Inferencecited by 1
  34. 2026Entropy-Constrained Adaptive Stochastic Quantizationcited by 1
  35. 2026Block-Sphere Vector Quantizationcited by 0
  36. 2026SSTQ:Privacy-Preserving Vector Quantization via Subsampled Stochastic TurboQuantcited by 0
  37. 2026Exact Minimax One-Bit Unbiased Compression: Heavy-Tail Necessity and Finite-Randomness Approximationcited by 0
  38. 2026TORQUE: Optimizing What (not) to Quantize Before and After Rotationcited by 0

50 most cited of 52 All 52 (JSON)