LogAvgExp Provides a Principled and Performant Global Pooling Operator
Not on the map. Why?
Abstract (arXiv)
We seek to improve the pooling operation in neural networks, by applying a more theoretically justified operator. We demonstrate that LogSumExp provides a natural OR operator for logits. When one corrects for the number of elements inside the pooling operator, this becomes LogAvgExp := log(mean(exp(x))). By introducing a single temperature parameter, LogAvgExp smoothly transitions from the max of its operands to the mean (found at the limiting cases t → 0^+ and t → +∞). We experimentally tested LogAvgExp, both with and without a learnable temperature parameter, in a variety of deep neural network architectures for computer vision.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 21 of its 40 references that are in the map. First posted on arXiv 2 Nov 2021.
Builds on
21 of 40 references.
References · 21
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2016Densely Connected Convolutional Networkscited by 6,720
- 2017Squeeze-and-Excitation Networkscited by 4,789
- 2014Going Deeper with Convolutionscited by 4,241
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
- 2016Wide Residual Networkscited by 3,070
- 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
- 2018Self-Attention Generative Adversarial Networkscited by 809
- 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 772
- 2018AutoAugment: Learning Augmentation Policies from Datacited by 730
- 2017Fine-tuning CNN Image Retrieval with No Human Annotationcited by 499
9 more
- 2018Bag of Tricks for Image Classification with Convolutional Neural Networkscited by 431
- 2016Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentationcited by 198
- 2016Deep Pyramidal Residual Networkscited by 163
- 2019Mish: A Self Regularized Non-Monotonic Activation Functioncited by 141
- 2016Revisiting Multiple Instance Neural Networkscited by 119
- 2018Fix your classifier: the marginal value of training the last weight layercited by 43
- 2014Fractional Max-Poolingcited by 35
- 2018Detail-Preserving Pooling in Deep Networkscited by 26
- 2015Deep SimNetscited by 10
Built on by
Citing papers per year: 2022: 1; 2023: 1.
2 papers.