mlsanityData to 7 Oct 2026

Iterative Teaching by Label Synthesis

arXiv 2110.14432 ↗First posted 27 Oct 2021cs.LG cs.AI cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

In this paper, we consider the problem of iterative machine teaching, where a teacher provides examples sequentially based on the current iterative learner. In contrast to previous methods that have to scan over the entire pool and select teaching examples from it in each iteration, we propose a label synthesis teaching framework where the teacher randomly selects input teaching examples (e.g., images) and then synthesizes suitable outputs (e.g., labels) for them. We show that this framework can avoid costly example selection while still provably achieving exponential teachability. We propose multiple novel teaching algorithms in this framework. Finally, we empirically demonstrate the value of our framework.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 64 of its 106 references that are in the map. First posted on arXiv 27 Oct 2021.

Builds on

64 of 106 references.

References · 64

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2016Densely Connected Convolutional Networkscited by 6,691
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  9. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  10. 2019When Does Label Smoothing Help?cited by 985
  11. 2017SphereFace: Deep Hypersphere Embedding for Face Recognitioncited by 733
  12. 2016Learning to learn by gradient descent by gradient descentcited by 616
38 more
  1. 2019Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processingcited by 411
  2. 2016Dynamic Filter Networkscited by 318
  3. 2016Large-Margin Softmax Loss for Convolutional Neural Networkscited by 296
  4. 2018Additive Margin Softmax for Face Verificationcited by 275
  5. 2015Unifying distillation and privileged informationcited by 218
  6. 2016Learning to Optimizecited by 195
  7. 2019Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Ratescited by 119
  8. 2016Towards better decoding and language model integration in sequence to sequence modelscited by 100
  9. 2014Stochastic Optimization with Importance Samplingcited by 91
  10. 2020Adaptive Reward-Poisoning Attacks against Reinforcement Learningcited by 86
  11. 2020Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learningcited by 81
  12. 2016Coresets for Scalable Bayesian Logistic Regressioncited by 79
  13. 2021Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Bettercited by 77
  14. 2018Learning towards Minimum Hyperspherical Energycited by 75
  15. 2019Policy Poisoning in Batch Reinforcement Learning and Controlcited by 70
  16. 2018An Overview of Machine Teachingcited by 63
  17. 2017Deep Hyperspherical Learningcited by 55
  18. 2020Orthogonal Over-Parameterized Trainingcited by 44
  19. 2016DisturbLabel: Regularizing CNN on the Loss Layercited by 41
  20. 2017Iterative Machine Teachingcited by 40
  21. 2018Learning to Teach with Dynamic Loss Functionscited by 40
  22. 2021Learning with Hyperspherical Uniformitycited by 35
  23. 2014Near-Optimally Teaching the Crowd to Classifycited by 32
  24. 2018Teaching Categories to Human Learners with Visual Explanationscited by 30
  25. 2019Angular Visual Hardnesscited by 29
  26. 2019Regularizing Neural Networks via Minimizing Hyperspherical Energycited by 25
  27. 2018Training Set Debugging Using Trusted Itemscited by 24
  28. 2019Interactive Teaching Algorithms for Inverse Reinforcement Learningcited by 24
  29. 2019A mathematical theory of cooperative communicationcited by 23
  30. 2015The Teaching Dimension of Linear Learnerscited by 19
  31. 2017Towards Black-box Iterative Machine Teachingcited by 19
  32. 2018Teaching Inverse Reinforcement Learners via Features and Demonstrationscited by 19
  33. 2013Machine Teaching for Bayesian Learners in the Exponential Familycited by 18
  34. 2019Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraintscited by 18
  35. 2018Teaching Multiple Concepts to a Forgetful Learnercited by 17
  36. 2018Understanding the Role of Adaptivity in Machine Teaching: The Case of Version Space Learnerscited by 16
  37. 2019Neural Similarity Learningcited by 13
  38. 2015Becoming the Expert - Interactive Multi-Class Machine Teachingcited by 12

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 4; 2023: 4; 2024: 1.

10 papers.

  1. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  2. 2021Multi-label Iterated Learning for Image Classification with Label Ambiguitycited by 19
  3. 2022On the Informativeness of Supervision Signalscited by 12
  4. 2023Nonparametric Iterative Machine Teachingcited by 9
  5. 2022Iterative Teaching by Data Hallucinationcited by 8
  6. 2022Human-in-the-Loop Mixupcited by 8
  7. 2023Nonparametric Teaching for Multiple Learnerscited by 7
  8. 2024Representational Alignment Supports Effective Machine Teachingcited by 3
  9. 2022One-shot Machine Teaching: Cost Very Few Examples to Converge Fastercited by 0
  10. 2023Optimally Teaching a Linear Behavior Cloning Agentcited by 0