mlsanityData to 7 Oct 2026

CSCO: Connectivity Search of Convolutional Operators

arXiv 2404.17152 ↗First posted 26 Apr 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Exploring dense connectivity of convolutional operators establishes critical "synapses" to communicate feature vectors from different levels and enriches the set of transformations on Computer Vision applications. Yet, even with heavy-machinery approaches such as Neural Architecture Search (NAS), discovering effective connectivity patterns requires tremendous efforts due to either constrained connectivity design space or a sub-optimal exploration process induced by an unconstrained search space. In this paper, we propose CSCO, a novel paradigm that fabricates effective connectivity of convolutional operators with minimal utilization of existing design motifs and further utilizes the discovered wiring to construct high-performing ConvNets. CSCO guides the exploration via a neural predictor as a surrogate of the ground-truth performance. We introduce Graph Isomorphism as data augmentation to improve sample efficiency and propose a Metropolis-Hastings Evolutionary Search (MH-ES) to evade locally optimal architectures and advance search quality. Results on ImageNet show ~0.6% performance improvement over hand-crafted and NAS-crafted dense connectivity. Our code is publicly available.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 52 references that are in the map. First posted on arXiv 26 Apr 2024.

Builds on

38 of 52 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  7. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  9. 2016Wide Residual Networkscited by 3,057
  10. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  11. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  12. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
26 more
  1. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  2. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  3. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  4. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  5. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  6. 2017Progressive Neural Architecture Searchcited by 483
  7. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  8. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  9. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 192
  10. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 175
  11. 2017CondenseNet: An Efficient DenseNet using Learned Group Convolutionscited by 121
  12. 2019Neural Predictor for Neural Architecture Searchcited by 116
  13. 2020BRP-NAS: Prediction-based NAS using GCNscited by 109
  14. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  15. 2019Exploring Randomly Wired Neural Networks for Image Recognitioncited by 103
  16. 2019Discovering Neural Wiringscited by 56
  17. 2019Densely Connected Search Space for More Flexible Neural Architecture Searchcited by 52
  18. 2020Geometry-Aware Gradient Algorithms for Neural Architecture Searchcited by 40
  19. 2020Neural Architecture Generator Optimizationcited by 30
  20. 2021Stronger NAS with Weaker Predictorscited by 28
  21. 2020Local Search is a Remarkably Strong Baseline for Neural Architecture Searchcited by 15
  22. 2021Homogeneous Architecture Augmentation for Neural Predictorcited by 14
  23. 2016Local Search Yields a PTAS for k-Means in Doubling Metricscited by 13
  24. 2020NASGEM: Neural Architecture Search via Graph Embedding Methodcited by 9
  25. 2020Understanding the wiring evolution in differentiable neural architecture searchcited by 3
  26. 2019AutoShrink: A Topology-aware NAS for Discovering Efficient Neural Architecturecited by 2

Built on by

Not cited yet.