mlsanityData to 7 Oct 2026

Compact Neural Networks via Stacking Designed Basic Units

arXiv 2205.01508 ↗First posted 3 May 2022cs.CV cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Unstructured pruning has the limitation of dealing with the sparse and irregular weights. By contrast, structured pruning can help eliminate this drawback but it requires complex criterion to determine which components to be pruned. To this end, this paper presents a new method termed TissueNet, which directly constructs compact neural networks with fewer weight parameters by independently stacking designed basic units, without requiring additional judgement criteria anymore. Given the basic units of various architectures, they are combined and stacked in a certain form to build up compact neural networks. We formulate TissueNet in diverse popular backbones for comparison with the state-of-the-art pruning methods on different benchmark datasets. Moreover, two new metrics are proposed to evaluate compression performance. Experiment results show that TissueNet can achieve comparable classification accuracy while saving up to around 80% FLOPs and 89.7% parameters. That is, stacking basic units provides a new promising way for network compression.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 56 references that are in the map. First posted on arXiv 3 May 2022.

Builds on

40 of 56 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2017Mask R-CNNcited by 7,259
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  8. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  9. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  10. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  11. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  12. 2014FitNets: Hints for Thin Deep Netscited by 1,619
28 more
  1. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  2. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  3. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  4. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  5. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  6. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  7. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  8. 2013Network In Networkcited by 600
  9. 2020What is the State of Neural Network Pruning?cited by 578
  10. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  11. 2019Importance Estimation for Neural Network Pruningcited by 531
  12. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
  13. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  14. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
  15. 2016Dynamic Network Surgery for Efficient DNNscited by 233
  16. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  17. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 226
  18. 2018Discrimination-aware Channel Pruning for Deep Neural Networkscited by 171
  19. 2017Soft Weight-Sharing for Neural Network Compressioncited by 99
  20. 2018ESPNetv2: A Light-weight, Power Efficient, and General Purpose Convolutional Neural Networkcited by 90
  21. 2019Network Pruning via Transformable Architecture Searchcited by 80
  22. 2019CARS: Continuous Evolution for Efficient Neural Architecture Searchcited by 79
  23. 2021CHIP: CHannel Independence-based Pruning for Compact Neural Networkscited by 76
  24. 2019Knowledge Distillation via Route Constrained Optimizationcited by 73
  25. 2020Discrimination-aware Network Pruning for Deep Model Compressioncited by 50
  26. 2019All You Need is a Few Shifts: Designing Efficient Convolutional Neural Networks for Image Classificationcited by 38
  27. 2018Constructing Fast Network through Deconstruction of Convolutioncited by 24
  28. 2018A novel channel pruning method for deep neural network compressioncited by 8

Built on by

Not cited yet.