# Multitask learning over graphs: An Approach for Distributed, Streaming Machine Learning

arXiv: https://arxiv.org/abs/2001.02112 · HTML: https://mlsanity.com/paper/2001.02112/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jan 2020.

## Built on by (most cited first, 26 of 26)

- [Opportunities of Federated Learning in Connected, Cooperative and Automated Industrial Systems](https://mlsanity.com/paper/2101.03367/) (2021)
- [Networked Signal and Information Processing](https://mlsanity.com/paper/2210.13767/) (2022)
- [Clustered Federated Learning via Generalized Total Variation Minimization](https://mlsanity.com/paper/2105.12769/) (2021)
- [Quantization for decentralized learning under subspace constraints](https://mlsanity.com/paper/2209.07821/) (2022)
- [Enforcing Privacy in Distributed Learning with Performance Guarantees](https://mlsanity.com/paper/2301.06412/) (2023)
- [Differential error feedback for communication-efficient decentralized learning](https://mlsanity.com/paper/2406.18418/) (2024)
- [Compressed Regression over Adaptive Networks](https://mlsanity.com/paper/2304.03638/) (2023)
- [Distributed Associative Memory via Online Convex Optimization](https://mlsanity.com/paper/2509.22321/) (2025)
- [Competing Adaptive Networks](https://mlsanity.com/paper/2103.15664/) (2021)
- [Collaborative likelihood-ratio estimation over graphs](https://mlsanity.com/paper/2205.14461/) (2022)
- [On the Energy and Communication Efficiency Tradeoffs in Federated and Multi-Task Learning](https://mlsanity.com/paper/2212.01049/) (2022)
- [High-Probability Convergence Guarantees of Decentralized SGD](https://mlsanity.com/paper/2510.06141/) (2025)
- [Dencentralized learning in the presence of low-rank noise](https://mlsanity.com/paper/2203.09810/) (2022)
- [Design and Analysis of Robust Resilient Diffusion over Multi-Task Networks Against Byzantine Attacks](https://mlsanity.com/paper/2206.12749/) (2022)
- [Association Graph Learning for Multi-Task Classification with Category Shifts](https://mlsanity.com/paper/2210.04637/) (2022)
- [Exact Subspace Diffusion for Decentralized Multitask Learning](https://mlsanity.com/paper/2304.07358/) (2023)
- [Online Multi-Task Learning with Recursive Least Squares and Recursive Kernel Methods](https://mlsanity.com/paper/2308.01938/) (2023)
- [Heuristic Vision Pre-Training with Self-Supervised and Supervised Multi-Task Learning](https://mlsanity.com/paper/2310.07510/) (2023)
- [Asynchronous Message-Passing and Zeroth-Order Optimization Based Distributed Learning with a Use-Case in Resource Allocation in Communication Networks](https://mlsanity.com/paper/2311.04604/) (2023)
- [Distributed Continual Learning with CoCoA in High-dimensional Linear Regression](https://mlsanity.com/paper/2312.01795/) (2023)
- [Joint estimation of smooth graph signals from partial linear measurements](https://mlsanity.com/paper/2505.23240/) (2025)
- [Quantitative Error Feedback for Quantization Noise Reduction of Filtering over Graphs](https://mlsanity.com/paper/2506.01404/) (2025)
- [A non-smooth regularization framework for learning over multitask graphs](https://mlsanity.com/paper/2509.17728/) (2025)
- [Distributed Dynamic Associative Memory via Online Convex Optimization](https://mlsanity.com/paper/2511.23347/) (2025)
- [High-Probability Convergence in Decentralized Stochastic Optimization with Gradient Tracking](https://mlsanity.com/paper/2605.00281/) (2026)
- [Decentralized Multitask Learning over Learned Task Graphs](https://mlsanity.com/paper/2608.26989/) (2026)

## Abstract (arXiv)

The problem of learning simultaneously several related tasks has received considerable attention in several domains, especially in machine learning with the so-called multitask learning problem or learning to learn problem [1], [2]. Multitask learning is an approach to inductive transfer learning (using what is learned for one problem to assist in another problem) and helps improve generalization performance relative to learning each task separately by using the domain information contained in the training signals of related tasks as an inductive bias. Several strategies have been derived within this community under the assumption that all data are available beforehand at a fusion center. However, recent years have witnessed an increasing ability to collect data in a distributed and streaming manner. This requires the design of new strategies for learning jointly multiple tasks from streaming data over distributed (or networked) systems. This article provides an overview of multitask strategies for learning and adaptation over networks. The working hypothesis for these strategies is that agents are allowed to cooperate with each other in order to learn distinct, though related tasks. The article shows how cooperation steers the network limiting point and how different cooperation rules allow to promote different task relatedness models. It also explains how and when cooperation over multitask networks outperforms non-cooperative strategies.