mlsanityData to 7 Oct 2026

Resource-constrained knowledge diffusion processes inspired by human peer learning

arXiv 2312.00660 ↗First posted 1 Dec 2023cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We consider a setting where a population of artificial learners is given, and the objective is to optimize aggregate measures of performance, under constraints on training resources. The problem is motivated by the study of peer learning in human educational systems. In this context, we study natural knowledge diffusion processes in networks of interacting artificial learners. By `natural', we mean processes that reflect human peer learning where the students' internal state and learning process is mostly opaque, and the main degree of freedom lies in the formation of peer learning groups by a coordinator who can potentially evaluate the learners before assigning them to peer groups. Among else, we empirically show that such processes indeed make effective use of the training resources, and enable the design of modular neural models that have the capacity to generalize without being prone to overfitting noisy labels.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 44 references that are in the map. First posted on arXiv 1 Dec 2023.

Builds on

21 of 44 references.

References · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  5. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  6. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
  7. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  8. 2017A Closer Look at Memorization in Deep Networkscited by 936
  9. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
  10. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 580
  11. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  12. 2019SGD on Neural Networks Learns Functions of Increasing Complexitycited by 203
9 more
  1. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 193
  2. 2021Assessing Generalization of SGD via Disagreementcited by 116
  3. 2020When Do Curricula Work?cited by 105
  4. 2016Decentralized Collaborative Learning of Personalized Models over Networkscited by 73
  5. 2016CoCoA: A General Framework for Communication-Efficient Distributed Optimizationcited by 69
  6. 2017Personalized and Private Peer-to-Peer Machine Learningcited by 54
  7. 2020Distributional Generalization: A New Kind of Generalizationcited by 41
  8. 2022Rare Gems: Finding Lottery Tickets at Initializationcited by 34
  9. 2022Find Your Friends: Personalized Federated Learning with the Right Collaboratorscited by 10

Built on by

Not cited yet.