# Multi Layer Neural Networks as Replacement for Pooling Operations

arXiv: https://arxiv.org/abs/2006.06969 · HTML: https://mlsanity.com/paper/2006.06969/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2020.

## Built on by (most cited first, 8 of 8)

- [TEyeD: Over 20 million real-world eye images with Pupil, Eyelid, and Iris 2D and 3D Segmentations, 2D and 3D Landmarks, 3D Eyeball, Gaze Vector, and Eye Movement Types](https://mlsanity.com/paper/2102.02115/) (2021)
- [The Gaze and Mouse Signal as additional Source for User Fingerprints in Browser Applications](https://mlsanity.com/paper/2101.03793/) (2021)
- [A Multimodal Eye Movement Dataset and a Multimodal Eye Movement Segmentation Analysis](https://mlsanity.com/paper/2101.04318/) (2021)
- [1000 Pupil Segmentations in a Second using Haar Like Features and Statistical Learning](https://mlsanity.com/paper/2102.01921/) (2021)
- [Maximum and Leaky Maximum Propagation](https://mlsanity.com/paper/2105.10277/) (2021)
- [Tensor Normalization and Full Distribution Training](https://mlsanity.com/paper/2109.02345/) (2021)
- [GroupGazer: A Tool to Compute the Gaze per Participant in Groups with integrated Calibration to Map the Gaze Online to a Screen or Beamer Projection](https://mlsanity.com/paper/2201.07692/) (2022)
- [Technical Report: Combining knowledge from Transfer Learning during training and Wide Resnets](https://mlsanity.com/paper/2206.09697/) (2022)

## Abstract (arXiv)

Pooling operations, which can be calculated at low cost and serve as a linear or nonlinear transfer function for data reduction, are found in almost every modern neural network. Countless modern approaches have already tackled replacing the common maximum value selection and mean value operations, not to mention providing a function that allows different functions to be selected through changing parameters. Additional neural networks are used to estimate the parameters of these pooling functions.Consequently, pooling layers may require supplementary parameters to increase the complexity of the whole model. In this work, we show that one perceptron can already be used effectively as a pooling operation without increasing the complexity of the model. This kind of pooling allows for the integration of multi-layer neural networks directly into a model as a pooling operation by restructuring the data and, as a result, learnin complex pooling operations. We compare our approach to tensor convolution with strides as a pooling operation and show that our approach is both effective and reduces complexity. The restructuring of the data in combination with multiple perceptrons allows for our approach to be used for upscaling, which can then be utilized for transposed convolutions in semantic segmentation.