# Clustered Federated Learning via Generalized Total Variation Minimization

arXiv: https://arxiv.org/abs/2105.12769 · HTML: https://mlsanity.com/paper/2105.12769/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 79 references that are in the map.
- First posted on arXiv 26 May 2021.

## Builds on (most cited first, 23 of 23)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Personalized Federated Learning with Moreau Envelopes](https://mlsanity.com/paper/2006.08848/) (2020)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Community Detection and Stochastic Block Models](https://mlsanity.com/paper/1703.10146/) (2017)
- [Decentralized Federated Averaging](https://mlsanity.com/paper/2104.11375/) (2021)
- [CoCoA: A General Framework for Communication-Efficient Distributed Optimization](https://mlsanity.com/paper/1611.02189/) (2016)
- [Network Lasso: Clustering and Optimization in Large Graphs](https://mlsanity.com/paper/1507.00280/) (2015)
- [Optimal Algorithms for Non-Smooth Distributed Optimization in Networks](https://mlsanity.com/paper/1806.00291/) (2018)
- [Learning Mixtures of Gaussians in High Dimensions](https://mlsanity.com/paper/1503.00424/) (2015)
- [Convex Clustering: Model, Theoretical Guarantee and Efficient Algorithm](https://mlsanity.com/paper/1810.02677/) (2018)
- [Multitask learning over graphs: An Approach for Distributed, Streaming Machine Learning](https://mlsanity.com/paper/2001.02112/) (2020)
- [Local Graph Clustering with Network Lasso](https://mlsanity.com/paper/2004.12199/) (2020)
- [Vector-Valued Graph Trend Filtering with Non-Convex Penalties](https://mlsanity.com/paper/1905.12692/) (2019)
- [Semi-supervised Learning in Network-Structured Data via Total Variation Minimization](https://mlsanity.com/paper/1901.09838/) (2019)
- [Flow-Based Local Graph Clustering with Better Seed Set Inclusion](https://mlsanity.com/paper/1811.12280/) (2018)
- [Localized Linear Regression in Networked Data](https://mlsanity.com/paper/1903.11178/) (2019)
- [Inexact First-Order Primal-Dual Algorithms](https://mlsanity.com/paper/1803.10576/) (2018)
- [Federated Learning From Big Data Over Networks](https://mlsanity.com/paper/2010.14159/) (2020)
- [When is Network Lasso Accurate?](https://mlsanity.com/paper/1704.02107/) (2017)
- [A Short Introduction to Local Graph Clustering Methods and Software](https://mlsanity.com/paper/1810.07324/) (2018)
- [On the Duality between Network Flows and Network Lasso](https://mlsanity.com/paper/1910.01805/) (2019)

## Built on by (most cited first, 7 of 7)

- [Tackling Feature and Sample Heterogeneity in Decentralized Multi-Task Learning: A Sheaf-Theoretic Approach](https://mlsanity.com/paper/2502.01145/) (2025)
- [Generalized Oversampling for Learning from Imbalanced datasets and Associated Theory](https://mlsanity.com/paper/2308.02966/) (2023)
- [A survey on Clustered Federated Learning: Taxonomy, Analysis and Applications](https://mlsanity.com/paper/2501.17512/) (2025)
- [Analysis of Total Variation Minimization for Clustered Federated Learning](https://mlsanity.com/paper/2403.06298/) (2024)
- [Recovery Guarantees for Distributed-OMP](https://mlsanity.com/paper/2209.07230/) (2022)
- [Your Data, My Model: Learning Who Really Helps in Federated Learning](https://mlsanity.com/paper/2409.02064/) (2024)
- [Smoothing ADMM for Non-convex and Non-smooth Hierarchical Federated Learning](https://mlsanity.com/paper/2503.08869/) (2025)

## Abstract (arXiv)

We study optimization methods to train local (or personalized) models for decentralized collections of local datasets with an intrinsic network structure. This network structure arises from domain-specific notions of similarity between local datasets. Examples for such notions include spatio-temporal proximity, statistical dependencies or functional relations. Our main conceptual contribution is to formulate federated learning as generalized total variation (GTV) minimization. This formulation unifies and considerably extends existing federated learning methods. It is highly flexible and can be combined with a broad range of parametric models, including generalized linear models or deep neural networks. Our main algorithmic contribution is a fully decentralized federated learning algorithm. This algorithm is obtained by applying an established primal-dual method to solve GTV minimization. It can be implemented as message passing and is robust against inexact computations that arise from limited computational resources including processing time or bandwidth. Our main analytic contribution is an upper bound on the deviation between the local model parameters learnt by our algorithm and an oracle-based clustered federated learning method. This upper bound reveals conditions on the local models and the network structure of local datasets such that GTV minimization is able to pool (nearly) homogeneous local datasets.