mlsanityData to 13 Oct 2026

More is Less: A More Complicated Network with Less Inference Complexity

arXiv 1703.08651 ↗First posted 25 Mar 2017cs.CVcited by 66

Not on the map. Why?

Abstract (arXiv)

In this paper, we present a novel and general network structure towards accelerating the inference process of convolutional neural networks, which is more complicated in network structure yet with less inference complexity. The core idea is to equip each original convolutional layer with another low-cost collaborative layer (LCCL), and the element-wise multiplication of the ReLU outputs of these two parallel layers produces the layer-wise output. The combined layer is potentially more discriminative than the original convolutional layer, and its inference is faster for two reasons: 1) the zero cells of the LCCL feature maps will remain zero after element-wise multiplication, and thus it is safe to skip the calculation of the corresponding high-cost convolution in the original convolutional layer, 2) LCCL is very fast if it is implemented as a 1*1 convolution or only a single filter shared by all channels. Extensive experiments on the CIFAR-10, CIFAR-100 and ILSCRC-2012 benchmarks show that our proposed network structure can accelerate the inference process by 32% on average with negligible performance drop.

Key facts. Cited by 66 papers in the map (data through 13 Oct 2026). First posted on arXiv 25 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 28; 2022: 10; 2023: 16; 2024: 7; 2025: 3; 2026: 2.

66 papers.

Sort:

By area:

  1. 2021Dynamic Neural Networks: A Surveycited by 406
  2. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 263
  3. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 201
  4. 2024DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Modelscited by 98
  5. 2021Tensor Methods in Computer Vision and Deep Learningcited by 52
  6. 2021PnP-DETR: Towards Efficient Visual Analysis with Transformerscited by 52
  7. 2023Adaptive Rotated Convolution for Rotated Object Detectioncited by 51
  8. 2022Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Modelscited by 45
  9. 2021Manifold Regularized Dynamic Network Pruningcited by 43
  10. 2021Locally Sparse Neural Networks for Tabular Biomedical Datacited by 34
  11. 2021Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decompositioncited by 33
  12. 2021Skip-Convolutions for Efficient Video Processingcited by 32
38 more
  1. 2021Dual-side Sparse Tensor Corecited by 29
  2. 2021Locally Free Weight Sharing for Network Width Searchcited by 27
  3. 2022Spatial Pruned Sparse Convolution for Efficient 3D Object Detectioncited by 26
  4. 2023Latency-aware Unified Dynamic Networks for Efficient Image Recognitioncited by 26
  5. 2022Latency-aware Spatial-wise Dynamic Networkscited by 25
  6. 2021Fire Together Wire Together: A Dynamic Pruning Approach with Self-Supervised Mask Predictioncited by 24
  7. 2022Pruning Neural Networks via Coresets and Convex Geometry: Towards No Assumptionscited by 20
  8. 2021PFA: Privacy-preserving Federated Adaptation for Effective Model Personalizationcited by 18
  9. 2021Anytime Dense Prediction with Confidence Adaptivitycited by 18
  10. 2021A Unified Approach to Coreset Learningcited by 18
  11. 2021Efficient Classification of Very Large Images with Tiny Objectscited by 17
  12. 2022Efficient Stein Variational Inference for Reliable Distribution-lossless Network Pruningcited by 16
  13. 2023Computation-efficient Deep Learning for Computer Vision: A Surveycited by 14
  14. 2021Automated Deep Learning: Neural Architecture Search Is Not the Endcited by 13
  15. 2023Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inferencecited by 11
  16. 2023Accelerating Deep Neural Networks via Semi-Structured Activation Sparsitycited by 10
  17. 2024Jointly Training and Pruning CNNs via Learnable Agent Guidance and Alignmentcited by 9
  18. 2025Hadamard product in deep learning: Introduction, Advances and Challengescited by 9
  19. 2023Training Large Language Models Efficiently with Sparsity and Dataflowcited by 7
  20. 2021DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformerscited by 6
  21. 2021CATRO: Channel Pruning via Class-Aware Trace Ratio Optimizationcited by 6
  22. 2021Low-rank Tensor Decomposition for Compression of Convolutional Neural Networks Using Funnel Regularizationcited by 6
  23. 2021Compact and Optimal Deep Learning with Recurrent Parameter Generatorscited by 5
  24. 2021Dynamic Collective Intelligence Learning: Finding Efficient Sparse Model via Refined Gradients for Pruned Weightscited by 4
  25. 2021EBJR: Energy-Based Joint Reasoning for Adaptive Inferencecited by 4
  26. 2022Searching for Network Width with Bilaterally Coupled Networkcited by 4
  27. 2023Towards Optimal Compression: Joint Pruning and Quantizationcited by 4
  28. 2023Progressive Channel-Shrinking Networkcited by 4
  29. 2023Influence Function Based Second-Order Channel Pruning-Evaluating True Loss Changes For Pruning Is Possible Without Retrainingcited by 4
  30. 2023Data-Free Dynamic Compression of CNNs for Tractable Efficiencycited by 4
  31. 2024Slicing Mutual Information Generalization Bounds for Neural Networkscited by 4
  32. 2024CNN Mixture-of-Depthscited by 3
  33. 2023SpVOS: Efficient Video Object Segmentation with Triple Sparse Convolutioncited by 2
  34. 2024Pick-or-Mix: Dynamic Channel Sampling for ConvNetscited by 2
  35. 2021Evolving Transferable Neural Pruning Functionscited by 1
  36. 2021Exploring Gradient Flow Based Saliency for DNN Model Compressioncited by 1
  37. 2021i-SpaSP: Structured Neural Pruning via Sparse Signal Recoverycited by 1
  38. 2023Integrating Fairness and Model Pruning Through Bi-level Optimizationcited by 1

50 most cited of 66 All 66 (JSON)