mlsanityData to 7 Oct 2026

Consensus learning: A novel decentralised ensemble learning paradigm

arXiv 2402.16157 ↗First posted 25 Feb 2024cs.LG cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

The widespread adoption of large-scale machine learning models in recent years highlights the need for distributed computing for efficiency and scalability. This work introduces a novel distributed machine learning paradigm -- consensus learning -- which combines classical ensemble methods with consensus protocols deployed in peer-to-peer systems. These algorithms consist of two phases: first, participants develop their models and submit predictions for any new data inputs; second, the individual predictions are used as inputs for a communication phase, which is governed by a consensus protocol. Consensus learning ensures user data privacy, while also inheriting the safety measures against Byzantine attacks from the underlying consensus mechanism. We provide a detailed theoretical analysis for a particular consensus protocol and compare the performance of the consensus learning ensemble with centralised ensemble learning algorithms. The discussion is supplemented by various numerical simulations, which describe the robustness of the algorithms against Byzantine participants.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 92 references that are in the map. First posted on arXiv 25 Feb 2024.

Builds on

31 of 92 references.

References · 31

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  6. 2018Federated Learning with Non-IID Datacited by 1,099
  7. 2019Deep Leakage from Gradientscited by 1,004
  8. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  9. 2018LEAF: A Benchmark for Federated Settingscited by 780
  10. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  11. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 660
  12. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
19 more
  1. 2019Representation Learning for Dynamic Graphs: A Surveycited by 260
  2. 2017EMNIST: an extension of MNIST to handwritten letterscited by 212
  3. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 193
  4. 2019A Survey on Distributed Machine Learningcited by 187
  5. 2018Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysiscited by 154
  6. 2019One-Shot Federated Learningcited by 124
  7. 2021DENSE: Data-Free One-Shot Federated Learningcited by 100
  8. 2019Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfercited by 91
  9. 2020Meta-Learning Requires Meta-Augmentationcited by 57
  10. 2020Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning)cited by 54
  11. 2018Meta-Learning Update Rules for Unsupervised Representation Learningcited by 37
  12. 2024A Survey on Efficient Federated Learning Methods for Foundation Model Trainingcited by 29
  13. 2019Manipulation Attacks in Local Differential Privacycited by 23
  14. 2017Blockchain Consensus Protocols in the Wildcited by 6
  15. 2024On the Byzantine-Resilience of Distillation-Based Federated Learningcited by 5
  16. 2019Scalable and Probabilistic Leaderless BFT Consensus through Metastabilitycited by 4
  17. 2022Byzantine Fault Tolerance in Distributed Machine Learning : a Surveycited by 4
  18. 2024An Analysis of Avalanche Consensuscited by 2
  19. 2022When is Spring coming? A Security Analysis of Avalanche Consensuscited by 1

Built on by

Not cited yet.