# Federated Learning Using Variance Reduced Stochastic Gradient for Probabilistically Activated Agents

arXiv: https://arxiv.org/abs/2210.14362 · HTML: https://mlsanity.com/paper/2210.14362/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 25 Oct 2022.

## Builds on (most cited first, 12 of 12)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Adaptive Federated Learning in Resource Constrained Edge Computing Systems](https://mlsanity.com/paper/1804.05271/) (2018)
- [A Joint Learning and Communications Framework for Federated Learning over Wireless Networks](https://mlsanity.com/paper/1909.07972/) (2019)
- [Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies](https://mlsanity.com/paper/2010.01243/) (2020)
- [Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning](https://mlsanity.com/paper/2101.11203/) (2021)
- [Federated Learning for Smart Healthcare: A Survey](https://mlsanity.com/paper/2111.08834/) (2021)
- [Scheduling Policies for Federated Learning in Wireless Networks](https://mlsanity.com/paper/1908.06287/) (2019)
- [Towards Flexible Device Participation in Federated Learning](https://mlsanity.com/paper/2006.06954/) (2020)
- [Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimization](https://mlsanity.com/paper/2201.07912/) (2022)

## Built on by (most cited first, 8 of 8)

- [Reinforcement Learning from Delayed Observations via World Models](https://mlsanity.com/paper/2403.12309/) (2024)
- [First-Order Dynamic Optimization for Streaming Convex Costs](https://mlsanity.com/paper/2310.07925/) (2023)
- [Non-Convex Optimization in Federated Learning via Variance Reduction and Adaptive Learning](https://mlsanity.com/paper/2412.11660/) (2024)
- [FedMPDD: Communication-Efficient Federated Learning with Privacy Preservation Attributes via Projected Directional Derivative](https://mlsanity.com/paper/2512.20814/) (2025)
- [FedScalar: Federated Learning with Scalar Communication for Bandwidth-Constrained Networks](https://mlsanity.com/paper/2410.02260/) (2024)
- [Time-Varying Convex Optimization with O(n) Computational Complexity](https://mlsanity.com/paper/2410.15009/) (2024)
- [Adaptive Threshold-Driven Continuous Greedy Method for Scalable Submodular Optimization](https://mlsanity.com/paper/2604.03419/) (2026)
- [Scalar Federated Learning for Linear Quadratic Regulator](https://mlsanity.com/paper/2604.05088/) (2026)

## Abstract (arXiv)

This paper proposes an algorithm for Federated Learning (FL) with a two-layer structure that achieves both variance reduction and a faster convergence rate to an optimal solution in the setting where each agent has an arbitrary probability of selection in each iteration. In distributed machine learning, when privacy matters, FL is a functional tool. Placing FL in an environment where it has some irregular connections of agents (devices), reaching a trained model in both an economical and quick way can be a demanding job. The first layer of our algorithm corresponds to the model parameter propagation across agents done by the server. In the second layer, each agent does its local update with a stochastic and variance-reduced technique called Stochastic Variance Reduced Gradient (SVRG). We leverage the concept of variance reduction from stochastic optimization when the agents want to do their local update step to reduce the variance caused by stochastic gradient descent (SGD). We provide a convergence bound for our algorithm which improves the rate from O(frac1√K) to O(frac1K) by using a constant step-size. We demonstrate the performance of our algorithm using numerical examples.