# Learned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks

arXiv: https://arxiv.org/abs/1311.1780 · HTML: https://mlsanity.com/paper/1311.1780/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2013.

## Built on by (most cited first, 30 of 32)

- [AdaPool: Exponential Adaptive Pooling for Information-Retaining Downsampling](https://mlsanity.com/paper/2111.00772/) (2021)
- [Global Features are All You Need for Image Retrieval and Reranking](https://mlsanity.com/paper/2308.06954/) (2023)
- [Refining activation downsampling with SoftPool](https://mlsanity.com/paper/2101.00440/) (2021)
- [Learning strides in convolutional neural networks](https://mlsanity.com/paper/2202.01653/) (2022)
- [Temporal Lift Pooling for Continuous Sign Language Recognition](https://mlsanity.com/paper/2207.08734/) (2022)
- [QGNN: Value Function Factorisation with Graph Neural Networks](https://mlsanity.com/paper/2205.13005/) (2022)
- [A Survey on Hyperlink Prediction](https://mlsanity.com/paper/2207.02911/) (2022)
- [Self-Attentive Pooling for Efficient Deep Learning](https://mlsanity.com/paper/2209.07659/) (2022)
- [Comparison of Methods Generalizing Max- and Average-Pooling](https://mlsanity.com/paper/2103.01746/) (2021)
- [Self-Supervision Enhances Instance-based Multiple Instance Learning Methods in Digital Pathology: A Benchmark Study](https://mlsanity.com/paper/2505.01109/) (2025)
- [On the Memory Mechanism of Tensor-Power Recurrent Models](https://mlsanity.com/paper/2103.01521/) (2021)
- [Content-Adaptive Downsampling in Convolutional Neural Networks](https://mlsanity.com/paper/2305.09504/) (2023)
- [Generalised f-Mean Aggregation for Graph Neural Networks](https://mlsanity.com/paper/2306.13826/) (2023)
- [Function-Space Optimality of Neural Architectures with Multivariate Nonlinearities](https://mlsanity.com/paper/2310.03696/) (2023)
- [A General Framework for Robust G-Invariance in G-Equivariant Networks](https://mlsanity.com/paper/2310.18564/) (2023)
- [Learning to Pool in Graph Neural Networks for Extrapolation](https://mlsanity.com/paper/2106.06210/) (2021)
- [Mapping the Internet: Modelling Entity Interactions in Complex Heterogeneous Networks](https://mlsanity.com/paper/2104.09650/) (2021)
- [Generalizing Aggregation Functions in GNNs:High-Capacity GNNs via Nonlinear Neighborhood Aggregators](https://mlsanity.com/paper/2202.09145/) (2022)
- [Adaptive Spatial Goodness Encoding: Advancing and Scaling Forward-Forward Learning Without Backpropagation](https://mlsanity.com/paper/2509.12394/) (2025)
- [Regularized Optimal Transport Layers for Generalized Global Pooling Operations](https://mlsanity.com/paper/2212.06339/) (2022)
- [An application of Pixel Interval Down-sampling (PID) for dense tiny microorganism counting on environmental microorganism images](https://mlsanity.com/paper/2204.01341/) (2022)
- [Hierarchical Spherical CNNs with Lifting-based Adaptive Wavelets for Pooling and Unpooling](https://mlsanity.com/paper/2205.15571/) (2022)
- [Local and Global Context-and-Object-part-Aware Superpixel-based Data Augmentation for Deep Visual Recognition](https://mlsanity.com/paper/2512.00130/) (2025)
- [Ordinal Pooling](https://mlsanity.com/paper/2109.01561/) (2021)
- [Revisiting Global Pooling through the Lens of Optimal Transport](https://mlsanity.com/paper/2201.09191/) (2022)
- [YZR-net : Self-supervised Hidden representations Invariant to Transformations for profanity detection](https://mlsanity.com/paper/2211.15532/) (2022)
- [A Novel Dual-pooling Attention Module for UAV Vehicle Re-identification](https://mlsanity.com/paper/2306.14104/) (2023)
- [Malicious Internet Entity Detection Using Local Graph Inference](https://mlsanity.com/paper/2408.03287/) (2024)
- [Is Complete Labeling Necessary? Understanding Active Learning in Longitudinal Medical Imaging](https://mlsanity.com/paper/2511.18007/) (2025)
- [Beyond Weighted Summation: Learnable Nonlinear Aggregation Functions for Robust Artificial Neurons](https://mlsanity.com/paper/2603.19344/) (2026)

## Abstract (arXiv)

In this paper we propose and investigate a novel nonlinear unit, called Lp unit, for deep neural networks. The proposed Lp unit receives signals from several projections of a subset of units in the layer below and computes a normalized Lp norm. We notice two interesting interpretations of the Lp unit. First, the proposed unit can be understood as a generalization of a number of conventional pooling operators such as average, root-mean-square and max pooling widely used in, for instance, convolutional neural networks (CNN), HMAX models and neocognitrons. Furthermore, the Lp unit is, to a certain degree, similar to the recently proposed maxout unit (Goodfellow et al., 2013) which achieved the state-of-the-art object recognition results on a number of benchmark datasets. Secondly, we provide a geometrical interpretation of the activation function based on which we argue that the Lp unit is more efficient at representing complex, nonlinear separating boundaries. Each Lp unit defines a superelliptic boundary, with its exact shape defined by the order p. We claim that this makes it possible to model arbitrarily shaped, curved boundaries more efficiently by combining a few Lp units of different orders. This insight justifies the need for learning different orders for each unit in the model. We empirically evaluate the proposed Lp units on a number of datasets and show that multilayer perceptrons (MLP) consisting of the Lp units achieve the state-of-the-art results on a number of benchmark datasets. Furthermore, we evaluate the proposed Lp unit on the recently proposed deep recurrent neural networks (RNN).