mlsanityData to 14 Oct 2026

Enhancing the accuracies by performing pooling decisions adjacent to the output layer

arXiv 2303.05800 ↗First posted 10 Mar 2023cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Learning classification tasks of (2nx2n) inputs typically consist of ≤ n (2x2) max-pooling (MP) operators along the entire feedforward deep architecture. Here we show, using the CIFAR-10 database, that pooling decisions adjacent to the last convolutional layer significantly enhance accuracies. In particular, average accuracies of the advanced-VGG with m layers (A-VGGm) architectures are 0.936, 0.940, 0.954, 0.955, and 0.955 for m=6, 8, 14, 13, and 16, respectively. The results indicate A-VGG8s' accuracy is superior to VGG16s', and that the accuracies of A-VGG13 and A-VGG16 are equal, and comparable to that of Wide-ResNet16. In addition, replacing the three fully connected (FC) layers with one FC layer, A-VGG6 and A-VGG14, or with several linear activation FC layers, yielded similar accuracies. These significantly enhanced accuracies stem from training the most influential input-output routes, in comparison to the inferior routes selected following multiple MP decisions along the deep architecture. In addition, accuracies are sensitive to the order of the non-commutative MP and average pooling operators adjacent to the output layer, varying the number and location of training routes. The results call for the reexamination of previously proposed deep architectures and their accuracies by utilizing the proposed pooling strategy adjacent to the output layer.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 15 of its 36 references that are in the map. First posted on arXiv 10 Mar 2023.

Builds on

15 of 36 references.

References · 15

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2016Wide Residual Networkscited by 3,070
  5. 2014Deep Learning in Neural Networks: An Overviewcited by 770
  6. 2017Understanding the Effective Receptive Field in Deep Convolutional Neural Networkscited by 553
  7. 2021Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mappingcited by 48
  8. 2015A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extractioncited by 46
  9. 2016Nesterov's Accelerated Gradient and Momentum as approximations to Regularised Update Descentcited by 26
  10. 2021Layer Folding: Neural Network Depth Reduction using Activation Linearizationcited by 14
  11. 2018Towards Principled Design of Deep Convolutional Networks: Introducing SimpNetcited by 11
  12. 2020Brain experiments imply adaptation mechanisms which outperform common AI learning algorithmscited by 7
3 more
  1. 2022Efficient shallow learning as an alternative to deep learningcited by 6
  2. 2022Learning on tree architectures outperforms a convolutional feedforward networkcited by 4
  3. 2021Long anisotropic absolute refractory periods with rapid rise-times to reliable responsivenesscited by 2

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1.

3 papers.

  1. 2023Towards a universal mechanism for successful deep learningcited by 8
  2. 2023The mechanism underlying successful deep learningcited by 4
  3. 2024Role of Delay in Brain Dynamicscited by 1