mlsanityData to 7 Oct 2026

HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Search

arXiv 2102.11646 ↗First posted 23 Feb 2021cs.LG cs.AI cs.CVcited by 20

Not on the map. Why?

Abstract (arXiv)

Realistic use of neural networks often requires adhering to multiple constraints on latency, energy and memory among others. A popular approach to find fitting networks is through constrained Neural Architecture Search (NAS), however, previous methods enforce the constraint only softly. Therefore, the resulting networks do not exactly adhere to the resource constraint and their accuracy is harmed. In this work we resolve this by introducing Hard Constrained diffeRentiable NAS (HardCoRe-NAS), that is based on an accurate formulation of the expected resource requirement and a scalable search method that satisfies the hard constraint throughout the search. Our experiments show that HardCoRe-NAS generates state-of-the-art architectures, surpassing other NAS methods, while strictly satisfying the hard resource constraints without any tuning required.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 50 of its 71 references that are in the map. First posted on arXiv 23 Feb 2021.

Builds on

50 of 71 references.

References · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  8. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  9. 2017Squeeze-and-Excitation Networkscited by 4,763
  10. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  11. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  12. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
38 more
  1. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  2. 2019Searching for MobileNetV3cited by 1,984
  3. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  4. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  5. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  6. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  7. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  8. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  9. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  10. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  11. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
  12. 2020ResNeSt: Split-Attention Networkscited by 445
  13. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
  14. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  15. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 328
  16. 2018SNAS: Stochastic Neural Architecture Searchcited by 288
  17. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  18. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 200
  19. 2016FINN: A Framework for Fast, Scalable Binarized Neural Network Inferencecited by 141
  20. 2019FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Searchcited by 129
  21. 2020TResNet: High Performance GPU-Dedicated Architecturecited by 119
  22. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  23. 2019A Survey of Optimization Methods from a Machine Learning Perspectivecited by 90
  24. 2019Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hourscited by 87
  25. 2019Network Pruning via Transformable Architecture Searchcited by 80
  26. 2015On the Global Linear Convergence of Frank-Wolfe Optimization Variantscited by 76
  27. 2016Variance-Reduced and Projection-Free Stochastic Optimizationcited by 40
  28. 2020Faster Projection-free Online Learningcited by 39
  29. 2019XNAS: Neural Architecture Search with Expert Advicecited by 35
  30. 2018A Frank-Wolfe Framework for Efficient and Effective Adversarial Attackscited by 29
  31. 2019ASAP: Architecture Search, Anneal and Prunecited by 29
  32. 2020Deep Neural Network Training with Frank-Wolfecited by 22
  33. 2020TF-NAS: Rethinking Three Search Freedoms of Latency-Constrained Differentiable Neural Architecture Searchcited by 21
  34. 2020Knapsack Pruning with Inner Distillationcited by 20
  35. 2020How to Train Your Super-Net: An Analysis of Training Heuristics in Weight-Sharing NAScited by 17
  36. 2018Deep Frank-Wolfe For Neural Network Optimizationcited by 11
  37. 2020Projection-Free Adaptive Gradients for Large-Scale Optimizationcited by 6
  38. 2020Global Convergence of Frank Wolfe on One Hidden Layer Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 3; 2023: 5; 2024: 4; 2025: 2; 2026: 3.

20 papers.

Sort:

  1. 2023What Can We Learn From The Selective Prediction And Uncertainty Estimation Performance Of 523 Imagenet Classifierscited by 35
  2. 2022CNN Filter DB: An Empirical Investigation of Trained Convolutional Filterscited by 20
  3. 2022On Redundancy and Diversity in Cell-based Neural Architecture Searchcited by 18
  4. 2021Layer Folding: Neural Network Depth Reduction using Activation Linearizationcited by 14
  5. 2023PLiNIO: A User-Friendly Library of Gradient-based Methods for Complexity-aware DNN Optimizationcited by 9
  6. 2024PNAS-MOT: Multi-Modal Object Tracking with Pareto Neural Architecture Searchcited by 9
  7. 2023Enhancing Neural Architecture Search with Multiple Hardware Constraints for Deep Learning Model Deployment on Tiny IoT Devicescited by 7
  8. 2023Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture Searchcited by 5
  9. 2024PePR: Performance Per Resource Unit as a Metric to Promote Small-Scale Deep Learning in Medical Image Analysiscited by 4
  10. 2024FreeAugment: Data Augmentation Search Across All Degrees of Freedomcited by 3
  11. 2021BINAS: Bilinear Interpretable Neural Architecture Searchcited by 2
  12. 2026Model Compression with Exact Budget Constraints via Riemannian Manifoldscited by 2
8 more
  1. 2022Which models are innately best at uncertainty estimation?cited by 1
  2. 2025Predicting upcoming visual features during eye movements yields scene representations aligned with human visual cortexcited by 1
  3. 2021Conditional super-network weightscited by 0
  4. 2023Enabling Hard Constraints in Differentiable Neural Network and Accelerator Co-Explorationcited by 0
  5. 2024CompactFlowNet: Efficient Real-time Optical Flow Estimation on Mobile Devicescited by 0
  6. 2025MedNNS: Supernet-based Medical Task-Adaptive Neural Network Searchcited by 0
  7. 2026HERCULES: Hardware-Efficient, Robust, Continual Learning Neural Architecture Searchcited by 0
  8. 2026ShellfishNet: A Domain-Specific Benchmark for Visual Recognition of Marine Molluscscited by 0