# A non-smooth regularization framework for learning over multitask graphs

arXiv: https://arxiv.org/abs/2509.17728 · HTML: https://mlsanity.com/paper/2509.17728/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 46 references that are in the map.
- First posted on arXiv 22 Sep 2025.

## Builds on (most cited first, 20 of 20)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Towards Personalized Federated Learning](https://mlsanity.com/paper/2103.00710/) (2021)
- [Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication](https://mlsanity.com/paper/1902.00340/) (2019)
- [Federated Evaluation of On-device Personalization](https://mlsanity.com/paper/1910.10252/) (2019)
- [Survey of Personalization Techniques for Federated Learning](https://mlsanity.com/paper/2003.08673/) (2020)
- [Network Lasso: Clustering and Optimization in Large Graphs](https://mlsanity.com/paper/1507.00280/) (2015)
- [Multitask learning over graphs: An Approach for Distributed, Streaming Machine Learning](https://mlsanity.com/paper/2001.02112/) (2020)
- [Proximity Without Consensus in Online Multi-Agent Optimization](https://mlsanity.com/paper/1606.05578/) (2016)
- [Learning over Multitask Graphs -- Part I: Stability Analysis](https://mlsanity.com/paper/1805.08535/) (2018)
- [Multitask Diffusion Adaptation over Networks](https://mlsanity.com/paper/1311.4894/) (2013)
- [Adaptation and learning over networks under subspace constraints -- Part I: Stability Analysis](https://mlsanity.com/paper/1905.08750/) (2019)
- [Distributed Optimization With Local Domains: Applications in MPC and Network Flows](https://mlsanity.com/paper/1305.1885/) (2013)
- [Distributed Coupled Multi-Agent Stochastic Optimization](https://mlsanity.com/paper/1712.08817/) (2017)
- [Distributed Diffusion-Based LMS for Node-Specific Adaptive Parameter Estimation](https://mlsanity.com/paper/1408.3354/) (2014)
- [Diffusion LMS for Multitask Problems with Local Linear Equality Constraints](https://mlsanity.com/paper/1610.02943/) (2016)
- [Proximal Multitask Learning over Networks with Sparsity-inducing Coregularization](https://mlsanity.com/paper/1509.01360/) (2015)

## Abstract (arXiv)

In this work, we consider learning over multitask graphs, where each agent aims to estimate its own parameter vector. Although agents seek distinct objectives, collaboration among them can be beneficial in scenarios where relationships between tasks exist. Among the various approaches to promoting relationships between tasks and, consequently, enhancing collaboration between agents, one notable method is regularization. While previous multitask learning studies have focused on smooth regularization to enforce graph smoothness, this work explores non-smooth regularization techniques that promote sparsity, making them particularly effective in encouraging piecewise constant transitions on the graph. We begin by formulating a global regularized optimization problem, which involves minimizing the aggregate sum of individual costs, regularized by a general non-smooth term designed to promote piecewise-constant relationships between the tasks of neighboring agents. Based on the forward-backward splitting strategy, we propose a decentralized learning approach that enables efficient solutions to the regularized optimization problem. Then, under convexity assumptions on the cost functions and co-regularization, we establish that the proposed approach converges in the mean-square-error sense within O(μ) of the optimal solution of the globally regularized cost. For broader applicability and improved computational efficiency, we also derive closed-form expressions for commonly used non-smooth (and, possibly, non-convex) regularizers, such as the weighted sum of the ℓ0-norm, ℓ1-norm, and elastic net regularization. Finally, we illustrate both the theoretical findings and the effectiveness of the approach through simulations.