# Self-organizing Democratized Learning: Towards Large-scale Distributed Learning Systems

arXiv: https://arxiv.org/abs/2007.03278 · HTML: https://mlsanity.com/paper/2007.03278/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jul 2020.

## Built on by (most cited first, 14 of 14)

- [Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matching](https://mlsanity.com/paper/2203.12285/) (2022)
- [FedICT: Federated Multi-task Distillation for Multi-access Edge Computing](https://mlsanity.com/paper/2301.00389/) (2023)
- [FedMEKT: Distillation-based Embedding Knowledge Transfer for Multimodal Federated Learning](https://mlsanity.com/paper/2307.13214/) (2023)
- [Multi-Edge Server-Assisted Dynamic Federated Learning with an Optimized Floating Aggregation Point](https://mlsanity.com/paper/2203.13950/) (2022)
- [Agglomerative Federated Learning: Empowering Larger Model Training via End-Edge-Cloud Collaboration](https://mlsanity.com/paper/2312.11489/) (2023)
- [A Contribution-based Device Selection Scheme in Federated Learning](https://mlsanity.com/paper/2203.05369/) (2022)
- [CDKT-FL: Cross-Device Knowledge Transfer using Proxy Dataset in Federated Learning](https://mlsanity.com/paper/2204.01542/) (2022)
- [Decentralized Federated Learning Over Imperfect Communication Channels](https://mlsanity.com/paper/2405.12894/) (2024)
- [Proximity-based Self-Federated Learning](https://mlsanity.com/paper/2407.12410/) (2024)
- [Implicit Model Specialization through DAG-based Decentralized Federated Learning](https://mlsanity.com/paper/2111.01257/) (2021)
- [Asynchronous Hierarchical Federated Learning](https://mlsanity.com/paper/2206.00054/) (2022)
- [Beyond Model Scale Limits: End-Edge-Cloud Federated Learning with Self-Rectified Knowledge Agglomeration](https://mlsanity.com/paper/2501.00693/) (2025)
- [Personalized Multi-tier Federated Learning](https://mlsanity.com/paper/2407.14251/) (2024)
- [Client Clustering Meets Knowledge Sharing: Enhancing Privacy and Robustness in Personalized Peer-to-Peer Learning](https://mlsanity.com/paper/2506.20413/) (2025)

## Abstract (arXiv)

Emerging cross-device artificial intelligence (AI) applications require a transition from conventional centralized learning systems towards large-scale distributed AI systems that can collaboratively perform complex learning tasks. In this regard, democratized learning (Dem-AI) lays out a holistic philosophy with underlying principles for building large-scale distributed and democratized machine learning systems. The outlined principles are meant to study a generalization in distributed learning systems that goes beyond existing mechanisms such as federated learning. Moreover, such learning systems rely on hierarchical self-organization of well-connected distributed learning agents who have limited and highly personalized data and can evolve and regulate themselves based on the underlying duality of specialized and generalized processes. Inspired by Dem-AI philosophy, a novel distributed learning approach is proposed in this paper. The approach consists of a self-organizing hierarchical structuring mechanism based on agglomerative clustering, hierarchical generalization, and corresponding learning mechanism. Subsequently, hierarchical generalized learning problems in recursive forms are formulated and shown to be approximately solved using the solutions of distributed personalized learning problems and hierarchical update mechanisms. To that end, a distributed learning algorithm, namely DemLearn is proposed. Extensive experiments on benchmark MNIST, Fashion-MNIST, FE-MNIST, and CIFAR-10 datasets show that the proposed algorithms demonstrate better results in the generalization performance of learning models in agents compared to the conventional FL algorithms. The detailed analysis provides useful observations to further handle both the generalization and specialization performance of the learning models in Dem-AI systems.