mlsanityData to 14 Oct 2026

Semi-Cyclic Stochastic Gradient Descent

arXiv 1904.10120 ↗First posted 23 Apr 2019cs.LG stat.MLcited by 40

Not on the map. Why?

Abstract (arXiv)

We consider convex SGD updates with a block-cyclic structure, i.e. where each cycle consists of a small number of blocks, each with many samples from a possibly different, block-specific, distribution. This situation arises, e.g., in Federated Learning where the mobile devices available for updates at different times during the day have different characteristics. We show that such block-cyclic structure can significantly deteriorate the performance of SGD, but propose a simple approach that allows prediction with the same performance guarantees as for i.i.d., non-cyclic, sampling.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 10; 2023: 12; 2024: 5; 2025: 5; 2026: 1.

40 papers.

Sort:

  1. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
  2. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 136
  3. 2021Fast Federated Learning in the Presence of Arbitrary Device Unavailabilitycited by 72
  4. 2021Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharingcited by 58
  5. 2022A Unified Analysis of Federated Learning with Arbitrary Client Participationcited by 45
  6. 2022Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learningcited by 33
  7. 2023On the Convergence of Federated Averaging with Cyclic Client Participationcited by 31
  8. 2022Architecture Agnostic Federated Learning for Neural Networkscited by 30
  9. 2022Federated Learning Under Intermittent Client Availability and Time-Varying Communication Constraintscited by 30
  10. 2023Federated Learning under Heterogeneous and Correlated Client Availabilitycited by 18
  11. 2023Federated Learning for Data Streamscited by 16
  12. 2022Anchor Sampling for Federated Learning with Partial Client Participationcited by 14
28 more
  1. 2023Performative Federated Learning: A Solution to Model-Dependent and Heterogeneous Distribution Shiftscited by 14
  2. 2024AdaFed: Fair Federated Learning via Adaptive Common Descent Directioncited by 13
  3. 2023BAFFLE: A Baseline of Backpropagation-Free Federated Learningcited by 12
  4. 2022Federated learning: Applications, challenges and future directionscited by 11
  5. 2025Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Accelerationcited by 9
  6. 2023Time-sensitive Learning for Heterogeneous Federated Edge Intelligencecited by 8
  7. 2023Mean Estimation with User-level Privacy under Data Heterogeneitycited by 8
  8. 2024Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysiscited by 8
  9. 2023On Convergence of Incremental Gradient for Non-Convex Smooth Functionscited by 7
  10. 2022On-Device Learning with Cloud-Coordinated Data Augmentation for Extreme Model Personalization in Recommender Systemscited by 6
  11. 2023Distributed Personalized Empirical Risk Minimizationcited by 6
  12. 2022Multi-Model Federated Learning with Provable Guaranteescited by 5
  13. 2022PersA-FL: Personalized Asynchronous Federated Learningcited by 5
  14. 2025Distributed Quasi-Newton Method for Fair and Fast Federated Learningcited by 5
  15. 2023Federated Learning Under Restricted User Availabilitycited by 4
  16. 2024Distributed Online Optimization with Stochastic Agent Availabilitycited by 3
  17. 2021An Energy Consumption Model for Electrical Vehicle Networks via Extended Federated-learningcited by 2
  18. 2024Efficient Language Model Architectures for Differentially Private Federated Learningcited by 2
  19. 2025Convergence of Agnostic Federated Averagingcited by 2
  20. 2025Efficient Federated Learning with Heterogeneous Data and Adaptive Dropoutcited by 2
  21. 2021Demystifying the Effects of Non-Independence in Federated Learningcited by 1
  22. 2023A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averagingcited by 1
  23. 2024Debiasing Federated Learning with Correlated Client Participationcited by 1
  24. 2021Federated Learning with Superquantile Aggregation for Heterogeneous Datacited by 0
  25. 2022Introducing Federated Learning into Internet of Things ecosystems -- preliminary considerationscited by 0
  26. 2023Tackling Computational Heterogeneity in FL: A Few Theoretical Insightscited by 0
  27. 2025Leveraging Label Potential for Enhanced Multimodal Emotion Recognitioncited by 0
  28. 2026FedeRage: Provably Convergent Agnostic Federated Learning under General Client Driftcited by 0