# LIP: Local Importance-based Pooling

arXiv: https://arxiv.org/abs/1908.04156 · HTML: https://mlsanity.com/paper/1908.04156/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Aug 2019.

## Built on by (most cited first, 30 of 30)

- [AdaPool: Exponential Adaptive Pooling for Information-Retaining Downsampling](https://mlsanity.com/paper/2111.00772/) (2021)
- [Refining activation downsampling with SoftPool](https://mlsanity.com/paper/2101.00440/) (2021)
- [Temporal Lift Pooling for Continuous Sign Language Recognition](https://mlsanity.com/paper/2207.08734/) (2022)
- [UniNet: Unified Architecture Search with Convolution, Transformer, and MLP](https://mlsanity.com/paper/2110.04035/) (2021)
- [Centroid Transformers: Learning to Abstract with Attention](https://mlsanity.com/paper/2102.08606/) (2021)
- [Breaking Immutable: Information-Coupled Prototype Elaboration for Few-Shot Object Detection](https://mlsanity.com/paper/2211.14782/) (2022)
- [Self-Attentive Pooling for Efficient Deep Learning](https://mlsanity.com/paper/2209.07659/) (2022)
- [Improving Shift Invariance in Convolutional Neural Networks with Translation Invariant Polyphase Sampling](https://mlsanity.com/paper/2404.07410/) (2024)
- [Content-Adaptive Downsampling in Convolutional Neural Networks](https://mlsanity.com/paper/2305.09504/) (2023)
- [CAPE: CAM as a Probabilistic Ensemble for Enhanced DNN Interpretation](https://mlsanity.com/paper/2404.02388/) (2024)
- [Meta-learning of Pooling Layers for Character Recognition](https://mlsanity.com/paper/2103.09528/) (2021)
- [Learnable Discrete Wavelet Pooling (LDW-Pooling) For Convolutional Networks](https://mlsanity.com/paper/2109.06638/) (2021)
- [Dynamic U-Net: Adaptively Calibrate Features for Abdominal Multi-organ Segmentation](https://mlsanity.com/paper/2403.07303/) (2024)
- [M2MRF: Many-to-Many Reassembly of Features for Tiny Lesion Segmentation in Fundus Images](https://mlsanity.com/paper/2111.00193/) (2021)
- [NOAH: Learning Pairwise Object Category Attentions for Image Classification](https://mlsanity.com/paper/2402.02377/) (2024)
- [Propagating Variational Model Uncertainty for Bioacoustic Call Label Smoothing](https://mlsanity.com/paper/2210.10526/) (2022)
- [Relevance-guided Audio Visual Fusion for Video Saliency Prediction](https://mlsanity.com/paper/2411.11454/) (2024)
- [DA-Font: Few-Shot Font Generation via Dual-Attention Hybrid Integration](https://mlsanity.com/paper/2509.16632/) (2025)
- [Local and Global Context-and-Object-part-Aware Superpixel-based Data Augmentation for Deep Visual Recognition](https://mlsanity.com/paper/2512.00130/) (2025)
- [Reconstructing Images of Two Adjacent Objects through Scattering Medium Using Generative Adversarial Network](https://mlsanity.com/paper/2107.11574/) (2021)
- [NoFADE: Analyzing Diminishing Returns on CO2 Investment](https://mlsanity.com/paper/2111.14059/) (2021)
- [LAP: An Attention-Based Module for Concept Based Self-Interpretation and Knowledge Injection in Convolutional Neural Networks](https://mlsanity.com/paper/2201.11808/) (2022)
- [UniNet: Unified Architecture Search with Convolution, Transformer, and MLP](https://mlsanity.com/paper/2207.05420/) (2022)
- [Dual Complementary Dynamic Convolution for Image Recognition](https://mlsanity.com/paper/2211.06163/) (2022)
- [MultiScale Probability Map guided Index Pooling with Attention-based learning for Road and Building Segmentation](https://mlsanity.com/paper/2302.09411/) (2023)
- [Decomposing and Coupling Saliency Map for Lesion Segmentation in Ultrasound Images](https://mlsanity.com/paper/2308.00947/) (2023)
- [The Road to On-board Change Detection: A Lightweight Patch-Level Change Detection Network via Exploring the Potential of Pruning and Pooling](https://mlsanity.com/paper/2310.10166/) (2023)
- [Dual-View Pyramid Pooling in Deep Neural Networks for Improved Medical Image Classification and Confidence Calibration](https://mlsanity.com/paper/2408.02906/) (2024)
- [Mitigating Context Bias in Domain Adaptation for Object Detection using Mask Pooling](https://mlsanity.com/paper/2505.18446/) (2025)
- [Redefining the Down-Sampling Scheme of U-Net for Precision Biomedical Image Segmentation](https://mlsanity.com/paper/2602.19412/) (2026)

## Abstract (arXiv)

Spatial downsampling layers are favored in convolutional neural networks (CNNs) to downscale feature maps for larger receptive fields and less memory consumption. However, for discriminative tasks, there is a possibility that these layers lose the discriminative details due to improper pooling strategies, which could hinder the learning process and eventually result in suboptimal models. In this paper, we present a unified framework over the existing downsampling layers (e.g., average pooling, max pooling, and strided convolution) from a local importance view. In this framework, we analyze the issues of these widely-used pooling layers and figure out the criteria for designing an effective downsampling layer. According to this analysis, we propose a conceptually simple, general, and effective pooling layer based on local importance modeling, termed as em Local Importance-based Pooling (LIP). LIP can automatically enhance discriminative features during the downsampling procedure by learning adaptive importance weights based on inputs. Experiment results show that LIP consistently yields notable gains with different depths and different architectures on ImageNet classification. In the challenging MS COCO dataset, detectors with our LIP-ResNets as backbones obtain a consistent improvement (≥ 1.4%) over the vanilla ResNets, and especially achieve the current state-of-the-art performance in detecting small objects under the single-scale testing scheme.