# FedHybrid: A Hybrid Primal-Dual Algorithm Framework for Federated Optimization

arXiv: https://arxiv.org/abs/2106.01279 · HTML: https://mlsanity.com/paper/2106.01279/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2021.

## Built on by (most cited first, 7 of 7)

- [Understanding Generalization of Federated Learning via Stability: Heterogeneity Matters](https://mlsanity.com/paper/2306.03824/) (2023)
- [FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneity](https://mlsanity.com/paper/2204.03529/) (2022)
- [Confederated Learning: Federated Learning with Decentralized Edge Servers](https://mlsanity.com/paper/2205.14905/) (2022)
- [Information-Theoretic Generalization Analysis for Topology-aware Heterogeneous Federated Edge Learning over Noisy Channels](https://mlsanity.com/paper/2310.16407/) (2023)
- [A-FedPD: Aligning Dual-Drift is All Federated Primal-Dual Learning Needs](https://mlsanity.com/paper/2409.18915/) (2024)
- [Attention, Anomalies! Handling Attention Layers in Unsupervised Federated Outlier Detection](https://mlsanity.com/paper/2608.04753/) (2026)
- [Federated Attention Autoencoders with a Stochastic Aggregation Scheme for Anomaly Detection](https://mlsanity.com/paper/2608.08906/) (2026)

## Abstract (arXiv)

We consider a multi-agent consensus optimization problem over a server-client (federated) network, where all clients are connected to a central server. Current distributed algorithms fail to capture the heterogeneity in clients' local computation capacities. Motivated by the generalized Method of Multipliers in centralized optimization, we derive an approximate Newton-type primal-dual method with a practical distributed implementation by utilizing the server-client topology. Then we propose a new primal-dual algorithm framework FedHybrid that allows different clients to perform various types of updates. Specifically, each client can choose to perform either gradient-type or Newton-type updates. We propose a novel analysis framework for primal-dual methods and obtain a linear convergence rate of FedHybrid for strongly convex functions, regardless of clients' choices of gradient-type or Newton-type updates. Numerical studies are provided to demonstrate the efficacy of our method in practice. To the best of our knowledge, this is the first hybrid algorithmic framework allowing heterogeneous local updates for distributed consensus optimization with a provable convergence and rate guarantee.