mlsanityData to 14 Oct 2026

Towards a Robust Differentiable Architecture Search under Label Noise

arXiv 2110.12197 ↗First posted 23 Oct 2021cs.LG cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Neural Architecture Search (NAS) is the game changer in designing robust neural architectures. Architectures designed by NAS outperform or compete with the best manual network designs in terms of accuracy, size, memory footprint and FLOPs. That said, previous studies focus on developing NAS algorithms for clean high quality data, a restrictive and somewhat unrealistic assumption. In this paper, focusing on the differentiable NAS algorithms, we show that vanilla NAS algorithms suffer from a performance loss if class labels are noisy. To combat this issue, we make use of the principle of information bottleneck as a regularizer. This leads us to develop a noise injecting operation that is included during the learning process, preventing the network from learning from noisy samples. Our empirical evaluations show that the noise injecting operation does not degrade the performance of the NAS algorithm if the data is indeed clean. In contrast, if the data is noisy, the architecture learned by our algorithm comfortably outperforms algorithms specifically equipped with sophisticated mechanisms to learn in the presence of label noise. In contrast to many algorithms designed to work in the presence of noisy labels, prior knowledge about the properties of the noise and its characteristics are not required for our algorithm.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 47 of its 66 references that are in the map. First posted on arXiv 23 Oct 2021.

Builds on

47 of 66 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Going Deeper with Convolutionscited by 4,241
  3. 2013Auto-Encoding Variational Bayescited by 4,053
  4. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  5. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  7. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  8. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  9. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
  10. 2016Deep Variational Information Bottleneckcited by 930
  11. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  12. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
35 more
  1. 2018Learning to Reweight Examples for Robust Deep Learningcited by 675
  2. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
  3. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 653
  4. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 503
  5. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 500
  6. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  7. 2017Large-Scale Evolution of Image Classifierscited by 348
  8. 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 328
  9. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  10. 2018Joint Optimization Framework for Learning with Noisy Labelscited by 293
  11. 2020Combating noisy labels by agreement: A joint training method with co-regularizationcited by 289
  12. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 279
  13. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  14. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 260
  15. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 260
  16. 2017Decoupling "when to update" from "how to update"cited by 230
  17. 2017Emergence of Invariance and Disentanglement in Deep Representationscited by 215
  18. 2019Probabilistic End-to-end Noise Correction for Learning with Noisy Labelscited by 210
  19. 2019Are Anchor Points Really Indispensable in Label-Noise Learning?cited by 208
  20. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  21. 2018Neural Architecture Optimizationcited by 167
  22. 2018Efficient Multi-objective Neural Architecture Search via Lamarckian Evolutioncited by 153
  23. 2016Information Dropout: Learning Optimal Representations Through Noisy Computationcited by 132
  24. 2019Learning From Noisy Labels By Regularized Estimation Of Annotator Confusioncited by 127
  25. 2018CurriculumNet: Weakly Supervised Learning from Large-Scale Web Imagescited by 126
  26. 2019AutoGAN: Neural Architecture Search for Generative Adversarial Networkscited by 94
  27. 2019DetNAS: Backbone Search for Object Detectioncited by 81
  28. 2017Toward Robustness against Label Noise in Training Deep Discriminative Neural Networkscited by 78
  29. 2020NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Searchcited by 64
  30. 2017Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimizationcited by 56
  31. 2016Noisy Activation Functionscited by 53
  32. 2020Are Labels Necessary for Neural Architecture Search?cited by 37
  33. 2016Convolutional Neural Fabricscited by 32
  34. 2020Rethinking Class Relations: Absolute-relative Supervised and Unsupervised Few-shot Learningcited by 31
  35. 2021REFINE: Random RangE FInder for Network Embeddingcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 0; 2026: 1.

2 papers.

  1. 2023Transductive Few-shot Learning with Prototype-based Label Propagation by Iterative Graph Refinementcited by 30
  2. 2026Neural Architecture Search of Sample Reweighting Networks for Complex Distribution Shiftcited by 0