mlsanityData to 14 Oct 2026

Learning from Mistakes -- A Framework for Neural Architecture Search

arXiv 2111.06353 ↗First posted 11 Nov 2021cs.LG cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Learning from one's mistakes is an effective human learning technique where the learners focus more on the topics where mistakes were made, so as to deepen their understanding. In this paper, we investigate if this human learning strategy can be applied in machine learning. We propose a novel machine learning method called Learning From Mistakes (LFM), wherein the learner improves its ability to learn by focusing more on the mistakes during revision. We formulate LFM as a three-stage optimization problem: 1) learner learns; 2) learner re-learns focusing on the mistakes, and; 3) learner validates its learning. We develop an efficient algorithm to solve the LFM problem. We apply the LFM framework to neural architecture search on CIFAR-10, CIFAR-100, and Imagenet. Experimental results strongly demonstrate the effectiveness of our model.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 34 of its 42 references that are in the map. First posted on arXiv 11 Nov 2021.

Builds on

34 of 42 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Densely Connected Convolutional Networkscited by 6,720
  3. 2014Going Deeper with Convolutionscited by 4,241
  4. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  7. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
  8. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
  9. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  10. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
  11. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  12. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
22 more
  1. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  2. 2018Learning to Reweight Examples for Robust Deep Learningcited by 675
  3. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  4. 2017Progressive Neural Architecture Searchcited by 487
  5. 2018SNAS: Stochastic Neural Architecture Searchcited by 289
  6. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 260
  7. 2017Hierarchical Representations for Efficient Architecture Searchcited by 191
  8. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  9. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 131
  10. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  11. 2019Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Datacited by 104
  12. 2020Stabilizing Differentiable Architecture Search via Perturbation-based Regularizationcited by 102
  13. 2020DARTS-: Robustly Stepping out of Performance Collapse Without Indicatorscited by 81
  14. 2017Practical Coreset Constructions for Machine Learningcited by 78
  15. 2020DrNAS: Dirichlet Neural Architecture Searchcited by 73
  16. 2020DSNAS: Direct Neural Architecture Search without Parameter Retrainingcited by 69
  17. 2019BayesNAS: A Bayesian Approach for Neural Architecture Searchcited by 68
  18. 2019Progressive DARTS: Bridging the Optimization Gap for NAS in the Wildcited by 40
  19. 2019ASAP: Architecture Search, Anneal and Prunecited by 29
  20. 2019Probabilistic Neural Architecture Searchcited by 20
  21. 2020Noisy Differentiable Architecture Searchcited by 16
  22. 2020Skillearn: Machine Learning Inspired by Humans' Learning Skillscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 2; 2025: 0; 2026: 1.

6 papers.

  1. 2022Betty: An Automatic Differentiation Library for Multilevel Optimizationcited by 27
  2. 2024BLO-SAM: Bi-level Optimization Based Overfitting-Preventing Finetuning of SAMcited by 8
  3. 2023Provably Convergent Federated Trilevel Learningcited by 6
  4. 2023Learning by Grouping: A Multilevel Optimization Framework for Improving Fairness in Classification without Losing Accuracycited by 0
  5. 2024Downstream Task Guided Masking Learning in Masked Autoencoders Using Multi-Level Optimizationcited by 0
  6. 2026LLM-Driven Multi-Turn Task-Oriented Dialogue Synthesis for Realistic Reasoningcited by 0