# The Effect of Training Parameters and Mechanisms on Decentralized Federated Learning based on MNIST Dataset

arXiv: https://arxiv.org/abs/2108.03508 · HTML: https://mlsanity.com/paper/2108.03508/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 25 references that are in the map.
- First posted on arXiv 7 Aug 2021.

## Builds on (most cited first, 12 of 12)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Federated Learning with Matched Averaging](https://mlsanity.com/paper/2002.06440/) (2020)
- [FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization](https://mlsanity.com/paper/1909.13014/) (2019)
- [Peer-to-peer Federated Learning on Graphs](https://mlsanity.com/paper/1901.11173/) (2019)
- [Federated Learning with Cooperating Devices: A Consensus Approach for Massive IoT Networks](https://mlsanity.com/paper/1912.13163/) (2019)
- [Distributed optimization over time-varying directed graphs](https://mlsanity.com/paper/1303.2289/) (2013)
- [Decentralized Federated Learning: A Segmented Gossip Approach](https://mlsanity.com/paper/1908.07782/) (2019)
- [Central Server Free Federated Learning over Single-sided Trust Social Networks](https://mlsanity.com/paper/1910.04956/) (2019)
- [Decentralized and Model-Free Federated Learning: Consensus-Based Distillation in Function Space](https://mlsanity.com/paper/2104.00352/) (2021)
- [L-FGADMM: Layer-Wise Federated Group ADMM for Communication Efficient Decentralized Deep Learning](https://mlsanity.com/paper/1911.03654/) (2019)
- [Fully Asynchronous Push-Sum With Growing Intercommunication Intervals](https://mlsanity.com/paper/1802.08634/) (2018)

## Abstract (arXiv)

Federated Learning is an algorithm suited for training models on decentralized data, but the requirement of a central "server" node is a bottleneck. In this document, we first introduce the notion of Decentralized Federated Learning (DFL). We then perform various experiments on different setups, such as changing model aggregation frequency, switching from independent and identically distributed (IID) dataset partitioning to non-IID partitioning with partial global sharing, using different optimization methods across clients, and breaking models into segments with partial sharing. All experiments are run on the MNIST handwritten digits dataset. We observe that those altered training procedures are generally robust, albeit non-optimal. We also observe failures in training when the variance between model weights is too large. The open-source experiment code is accessible through GitHubfootnoteCode was uploaded at urlhttps://github.com/zhzhang2018/DecentralizedFL.