mlsanityData to 7 Oct 2026

ADDS: Adaptive Differentiable Sampling for Robust Multi-Party Learning

arXiv 2110.15522 ↗First posted 29 Oct 2021cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Distributed multi-party learning provides an effective approach for training a joint model with scattered data under legal and practical constraints. However, due to the quagmire of a skewed distribution of data labels across participants and the computation bottleneck of local devices, how to build smaller customized models for clients in various scenarios while providing updates appliable to the central model remains a challenge. In this paper, we propose a novel adaptive differentiable sampling framework (ADDS) for robust and communication-efficient multi-party learning. Inspired by the idea of dropout in neural networks, we introduce a network sampling strategy in the multi-party setting, which distributes different subnets of the central model to clients for updating, and the differentiable sampling rates allow each client to extract optimal local architecture from the supernet according to its private data distribution. The approach requires minimal modifications to the existing multi-party learning structure, and it is capable of integrating local updates of all subnets into the supernet, improving the robustness of the central model. The proposed framework significantly reduces local computation and communication costs while speeding up the central model convergence, as we demonstrated through experiments on real-world datasets.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 39 of its 52 references that are in the map. First posted on arXiv 29 Oct 2021.

Builds on

39 of 52 references.

References · 39

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  6. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  7. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  8. 2018Federated Learning with Non-IID Datacited by 1,099
  9. 2019Relational Knowledge Distillationcited by 873
  10. 2019Towards Federated Learning at Scale: System Designcited by 826
  11. 2018LEAF: A Benchmark for Federated Settingscited by 780
  12. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
27 more
  1. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  2. 2017Federated Multi-Task Learningcited by 669
  3. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  4. 2018Rethinking the Value of Network Pruningcited by 595
  5. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  6. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  7. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  8. 2020Adaptive Personalized Federated Learningcited by 353
  9. 2018Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layerscited by 340
  10. 2020Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlookscited by 312
  11. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  12. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  13. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 241
  14. 2017EMNIST: an extension of MNIST to handwritten letterscited by 212
  15. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  16. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 178
  17. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 162
  18. 2017Revisiting Distributed Synchronous SGDcited by 142
  19. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  20. 2019Reliable Federated Learning for Mobile Networkscited by 92
  21. 2019Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruningcited by 89
  22. 2020DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocationcited by 47
  23. 2019Convergence of Learning Dynamics in Stackelberg Gamescited by 28
  24. 2018Asymptotic Soft Filter Pruning for Deep Convolutional Neural Networkscited by 27
  25. 2020On the Global Optimality of Model-Agnostic Meta-Learningcited by 21
  26. 2017Predicting Aesthetic Score Distribution through Cumulative Jensen-Shannon Divergencecited by 9
  27. 2019Transferring Knowledge Fragments for Learning Distance Metric from A Heterogeneous Domaincited by 5

Built on by

Not cited yet.