# Implicit Model Specialization through DAG-based Decentralized Federated Learning

arXiv: https://arxiv.org/abs/2111.01257 · HTML: https://mlsanity.com/paper/2111.01257/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 76 references that are in the map.
- First posted on arXiv 1 Nov 2021.

## Builds on (most cited first, 27 of 27)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Ditto: Fair and Robust Federated Learning Through Personalization](https://mlsanity.com/paper/2012.04221/) (2020)
- [Adaptive Personalized Federated Learning](https://mlsanity.com/paper/2003.13461/) (2020)
- [Three Approaches for Personalization with Applications to Federated Learning](https://mlsanity.com/paper/2002.10619/) (2020)
- [Personalized Federated Learning: A Meta-Learning Approach](https://mlsanity.com/paper/2002.07948/) (2020)
- [The Non-IID Data Quagmire of Decentralized Machine Learning](https://mlsanity.com/paper/1910.00189/) (2019)
- [Improving Federated Learning Personalization via Model Agnostic Meta Learning](https://mlsanity.com/paper/1909.12488/) (2019)
- [Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning](https://mlsanity.com/paper/1812.00535/) (2018)
- [Federated Evaluation of On-device Personalization](https://mlsanity.com/paper/1910.10252/) (2019)
- [Blockchained On-Device Federated Learning](https://mlsanity.com/paper/1808.03949/) (2018)
- [Decentralized Collaborative Learning of Personalized Models over Networks](https://mlsanity.com/paper/1610.05202/) (2016)
- [Variational Federated Multi-Task Learning](https://mlsanity.com/paper/1906.06268/) (2019)
- [Decentralized Federated Learning: A Segmented Gossip Approach](https://mlsanity.com/paper/1908.07782/) (2019)
- [Blockchain-Federated-Learning and Deep Learning Models for COVID-19 detection using CT Imaging](https://mlsanity.com/paper/2007.06537/) (2020)
- [Biscotti: A Ledger for Private and Secure Peer-to-Peer Machine Learning](https://mlsanity.com/paper/1811.09904/) (2018)
- [Self-organizing Democratized Learning: Towards Large-scale Distributed Learning Systems](https://mlsanity.com/paper/2007.03278/) (2020)
- [GoSGD: Distributed Optimization for Deep Learning with Gossip Exchange](https://mlsanity.com/paper/1804.01852/) (2018)
- [Distributed and Democratized Learning: Philosophy and Research Challenges](https://mlsanity.com/paper/2003.09301/) (2020)
- [Casper the Friendly Finality Gadget](https://mlsanity.com/paper/1710.09437/) (2017)
- [Scaling Nakamoto Consensus to Thousands of Transactions per Second](https://mlsanity.com/paper/1805.03870/) (2018)

## Built on by (most cited first, 4 of 4)

- [A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applications](https://mlsanity.com/paper/2403.01387/) (2024)
- [An Energy Optimized Specializing DAG Federated Learning based on Event Triggered Communication](https://mlsanity.com/paper/2209.12531/) (2022)
- [DAG-ACFL: Asynchronous Clustered Federated Learning based on DAG-DLT](https://mlsanity.com/paper/2308.13158/) (2023)
- [MultiConfederated Learning: Inclusive Non-IID Data handling with Decentralized Federated Learning](https://mlsanity.com/paper/2404.13421/) (2024)

## Abstract (arXiv)

Federated learning allows a group of distributed clients to train a common machine learning model on private data. The exchange of model updates is managed either by a central entity or in a decentralized way, e.g. by a blockchain. However, the strong generalization across all clients makes these approaches unsuited for non-independent and identically distributed (non-IID) data. We propose a unified approach to decentralization and personalization in federated learning that is based on a directed acyclic graph (DAG) of model updates. Instead of training a single global model, clients specialize on their local data while using the model updates from other clients dependent on the similarity of their respective data. This specialization implicitly emerges from the DAG-based communication and selection of model updates. Thus, we enable the evolution of specialized models, which focus on a subset of the data and therefore cover non-IID data better than federated learning in a centralized or blockchain-based setup. To the best of our knowledge, the proposed solution is the first to unite personalization and poisoning robustness in fully decentralized federated learning. Our evaluation shows that the specialization of models emerges directly from the DAG-based communication of model updates on three different datasets. Furthermore, we show stable model accuracy and less variance across clients when compared to federated averaging.