mlsanityData to 7 Oct 2026

Reciprocal Learning

arXiv 2408.06257 ↗First posted 12 Aug 2024stat.ML cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We demonstrate that a wide array of machine learning algorithms are specific instances of one single paradigm: reciprocal learning. These instances range from active learning over multi-armed bandits to self-training. We show that all these algorithms do not only learn parameters from data but also vice versa: They iteratively alter training data in a way that depends on the current model fit. We introduce reciprocal learning as a generalization of these algorithms using the language of decision theory. This allows us to study under what conditions they converge. The key is to guarantee that reciprocal learning contracts such that the Banach fixed-point theorem applies. In this way, we find that reciprocal learning algorithms converge at linear rates to an approximately optimal model under relatively mild assumptions on the loss function, if their predictions are probabilistic and the sample adaption is both non-greedy and either randomized or regularized. We interpret these findings and provide corollaries that relate them to specific active learning, self-training, and bandit algorithms.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 63 of its 146 references that are in the map. First posted on arXiv 12 Aug 2024.

Builds on

63 of 146 references.

References · 63

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2017Deep Bayesian Active Learning with Image Datacited by 812
  4. 2019Gradient based sample selection for online continual learningcited by 568
  5. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 547
  6. 2020A Survey of Deep Active Learningcited by 547
  7. 2020Dataset Condensation with Gradient Matchingcited by 510
  8. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 507
  9. 2019Introduction to Multi-Armed Banditscited by 475
  10. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  11. 2017A Tutorial on Thompson Samplingcited by 420
  12. 2023Scaling Data-Constrained Language Modelscited by 391
38 more
  1. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 354
  2. 2020Performative Predictioncited by 318
  3. 2020Amnesiac Machine Learningcited by 305
  4. 2021In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learningcited by 301
  5. 2018Online Learning: A Comprehensive Surveycited by 203
  6. 2014An Information-Theoretic Analysis of Thompson Samplingcited by 178
  7. 2023DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Modelscited by 121
  8. 2019Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Costcited by 114
  9. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  10. 2018Preference-based Online Learning with Dueling Bandits: A Surveycited by 100
  11. 2021Outside the Echo Chamber: Optimizing the Performative Riskcited by 95
  12. 2020Dynamic Regret of Convex and Smooth Functionscited by 90
  13. 2016Coresets for Scalable Bayesian Logistic Regressioncited by 79
  14. 2024No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performancecited by 75
  15. 2018On Coresets for Logistic Regressioncited by 65
  16. 2014Algorithms for multi-armed bandit problemscited by 63
  17. 2022"Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interactioncited by 63
  18. 2022Adaptive Second Order Coresets for Data-efficient Machine Learningcited by 60
  19. 2023Performative Prediction: Past and Futurecited by 52
  20. 2021Statistical Inference with M-Estimators on Adaptively Collected Datacited by 40
  21. 2021Self-Tuning for Data-Efficient Deep Learningcited by 39
  22. 2015A Survey on Contextual Multi-armed Banditscited by 34
  23. 2015Upper-Confidence-Bound Algorithms for Active Learning in Multi-Armed Banditscited by 29
  24. 2021A Closer Look at the Worst-case Behavior of Multi-armed Bandit Algorithmscited by 29
  25. 2022Performative Reinforcement Learningcited by 25
  26. 2019Adaptive Regret of Convex and Smooth Functionscited by 24
  27. 2019Stable-Predictive Optimistic Counterfactual Regret Minimizationcited by 21
  28. 2020Stopping criterion for active learning based on deterministic generalization boundscited by 20
  29. 2022Statistical Comparisons of Classifiers by Generalized Stochastic Dominancecited by 20
  30. 2022Training Subset Selection for Weak Supervisioncited by 19
  31. 2020Fixed Point Strategies in Data Sciencecited by 16
  32. 2022Information Gain Propagation: a new way to Graph Active Learning with Soft Labelscited by 16
  33. 2021Accounting for Gaussian Process Imprecision in Bayesian Optimizationcited by 13
  34. 2022Safe Active Learning for Multi-Output Gaussian Processescited by 13
  35. 2023Robust Statistical Comparison of Random Variables with Locally Varying Scale of Measurementcited by 13
  36. 2023Artificial Intelligence Index Report 2023cited by 13
  37. 2024Safe Active Learning for Time-Series Modeling with Gaussian Processescited by 12
  38. 2024Explaining Bayesian Optimization by Shapley Values Facilitates Human-AI Collaborationcited by 12

50 most cited of 63 All 63 (JSON)

Built on by

Not cited yet.