mlsanityData to 7 Oct 2026

Exploiting Correlations in Federated Learning: Opportunities and Practical Limitations

arXiv 2604.14751 ↗First posted 16 Apr 2026cs.IT cs.DC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The communication bottleneck in federated learning (FL) has spurred extensive research into techniques to reduce the volume of data exchanged between client devices and the central parameter server. In this paper, we systematically classify gradient and model compression schemes into three categories based on the type of correlations they exploit: structural, temporal, and spatial. We examine the sources of such correlations, propose quantitative metrics for measuring their magnitude, and reinterpret existing compression methods through this unified correlation-based framework. Our experimental studies demonstrate that the degrees of structural, temporal, and spatial correlations vary significantly depending on task complexity, model architecture, and algorithmic configurations. These findings suggest that algorithm designers should carefully evaluate correlation assumptions under specific deployment scenarios rather than assuming that they are always present. Motivated by these findings, we propose two adaptive compression designs that actively switch between different compression modes based on the measured correlation strength, and we evaluate their performance gains relative to conventional non-adaptive approaches. In summary, our unified taxonomy provides a clean and principled foundation for developing more effective and application-specific compression techniques for FL systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 54 references that are in the map. First posted on arXiv 16 Apr 2026.

Builds on

34 of 54 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  6. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  7. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  8. 2024GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projectioncited by 447
  9. 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 364
  10. 2017Empirical Analysis of the Hessian of Over-Parametrized Neural Networkscited by 283
  11. 2019Intrinsic dimension of data representations in deep neural networkscited by 276
  12. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 234
22 more
  1. 2018Gradient Descent Happens in a Tiny Subspacecited by 208
  2. 2017Gradient Sparsification for Communication-Efficient Distributed Optimizationcited by 199
  3. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 178
  4. 2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 140
  5. 2018ATOMO: Communication-efficient Learning via Atomic Sparsificationcited by 135
  6. 2021EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedbackcited by 131
  7. 2019Distributed Learning with Compressed Gradient Differencescited by 110
  8. 2024Flora: Low-Rank Adapters Are Secretly Gradient Compressorscited by 103
  9. 2018LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learningcited by 95
  10. 2019Accelerating Federated Learning via Momentum Gradient Descentcited by 75
  11. 2021Time-Correlated Sparsification for Communication-Efficient Federated Learningcited by 25
  12. 2018GradiVeQ: Vector Quantization for Bandwidth-Efficient Gradient Aggregation in Distributed CNN Trainingcited by 21
  13. 2021Learned Gradient Compression for Distributed Deep Learningcited by 20
  14. 2022Recycling Model Updates in Federated Learning: Are Gradient Subspaces Low-Rank?cited by 16
  15. 2021Faster Rates for Compressed Federated Learning with Client-Variance Reductioncited by 14
  16. 2022Distributed Learning With Sparsified Gradient Differencescited by 9
  17. 2022Time-Correlated Sparsification for Efficient Over-the-Air Model Aggregation in Wireless Federated Learningcited by 7
  18. 2023LASER: Linear Compression in Wireless Distributed Optimizationcited by 6
  19. 2021Communication-Efficient Federated Learning via Predictive Codingcited by 5
  20. 2025An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Datacited by 4
  21. 2021Compressing gradients by exploiting temporal correlation in momentum-SGDcited by 2
  22. 2024Temporal Predictive Coding for Gradient Compression in Distributed Learningcited by 2

Built on by

Not cited yet.