mlsanityData to 7 Oct 2026

Pick-or-Mix: Dynamic Channel Sampling for ConvNets

arXiv 2406.10935 ↗First posted 16 Jun 2024cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Channel pruning approaches for convolutional neural networks (ConvNets) deactivate the channels, statically or dynamically, and require special implementation. In addition, channel squeezing in representative ConvNets is carried out via 1x1 convolutions which dominates a large portion of computations and network parameters. Given these challenges, we propose an effective multi-purpose module for dynamic channel sampling, namely Pick-or-Mix (PiX), which does not require special implementation. PiX divides a set of channels into subsets and then picks from them, where the picking decision is dynamically made per each pixel based on the input activations. We plug PiX into prominent ConvNet architectures and verify its multi-purpose utilities. After replacing 1x1 channel squeezing layers in ResNet with PiX, the network becomes 25% faster without losing accuracy. We show that PiX allows ConvNets to learn better data representation than widely adopted approaches to enhance networks' representation power (e.g., SE, CBAM, AFF, SKNet, and DWP). We also show that PiX achieves state-of-the-art performance on network downscaling and dynamic channel pruning applications.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 35 of its 43 references that are in the map. First posted on arXiv 16 Jun 2024.

Builds on

35 of 43 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  8. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  9. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  10. 2017Squeeze-and-Excitation Networkscited by 4,763
  11. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  12. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
23 more
  1. 2016Pyramid Scene Parsing Networkcited by 2,472
  2. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
  3. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  4. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  5. 2020Designing Network Design Spacescited by 954
  6. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 520
  7. 2020ResNeSt: Split-Attention Networkscited by 445
  8. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  9. 2019GhostNet: More Features from Cheap Operationscited by 410
  10. 2019Selective Kernel Networkscited by 401
  11. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  12. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 313
  13. 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 172
  14. 2018Discrimination-aware Channel Pruning for Deep Neural Networkscited by 171
  15. 2018Dynamic Channel Pruning: Feature Boosting and Suppressioncited by 129
  16. 2020Attentional Feature Fusioncited by 115
  17. 2018Channel Gating Neural Networkscited by 80
  18. 2017More is Less: A More Complicated Network with Less Inference Complexitycited by 66
  19. 2021Manifold Regularized Dynamic Network Pruningcited by 43
  20. 2022PeRFception: Perception using Radiance Fieldscited by 19
  21. 2023Dynamic Structure Pruning for Compressing CNNscited by 5
  22. 2021Towards Deep Learning Assisted Autonomous UAVs for Manipulation Tasks in GPS-Denied Environmentscited by 3
  23. 2021Semi Supervised Deep Quick Instance Detection and Segmentationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 1.

2 papers.

  1. 2024Not All Prompts Are Made Equal: Prompt-based Pruning of Text-to-Image Diffusion Modelscited by 13
  2. 2026Mixture of Channel Experts: Static Sparse Supports with Input-Adaptive Mixing for Pointwise Projectionscited by 0