# Pixel-wise Attentional Gating for Parsimonious Pixel Labeling

arXiv: https://arxiv.org/abs/1805.01556 · HTML: https://mlsanity.com/paper/1805.01556/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 May 2018.

## Built on by (most cited first, 12 of 12)

- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Transformers in Self-Supervised Monocular Depth Estimation with Unknown Camera Intrinsics](https://mlsanity.com/paper/2202.03131/) (2022)
- [Multimodal Scale Consistency and Awareness for Monocular Self-Supervised Depth Estimation](https://mlsanity.com/paper/2103.02451/) (2021)
- [RayMVSNet++: Learning Ray-based 1D Implicit Fields for Accurate Multi-View Stereo](https://mlsanity.com/paper/2307.10233/) (2023)
- [Learning to Zoom and Unzoom](https://mlsanity.com/paper/2303.15390/) (2023)
- [Med-DANet: Dynamic Architecture Network for Efficient Medical Volumetric Segmentation](https://mlsanity.com/paper/2206.06575/) (2022)
- [Dynamic Frame Interpolation in Wavelet Domain](https://mlsanity.com/paper/2309.03508/) (2023)
- [SlowFormer: Universal Adversarial Patch for Attack on Compute and Energy Efficiency of Inference Efficient Vision Transformers](https://mlsanity.com/paper/2310.02544/) (2023)
- [Monocular Depth Estimation Primed by Salient Point Detection and Normalized Hessian Loss](https://mlsanity.com/paper/2108.11098/) (2021)
- [Switchable Representation Learning Framework with Self-compatibility](https://mlsanity.com/paper/2206.08289/) (2022)
- [Learning Positive-Incentive Point Sampling in Neural Implicit Fields for Object Pose Estimation](https://mlsanity.com/paper/2602.19937/) (2026)

## Abstract (arXiv)

To achieve parsimonious inference in per-pixel labeling tasks with a limited computational budget, we propose a Pixel-wise Attentional Gating unit (PAG) that learns to selectively process a subset of spatial locations at each layer of a deep convolutional network. PAG is a generic, architecture-independent, problem-agnostic mechanism that can be readily "plugged in" to an existing model with fine-tuning. We utilize PAG in two ways: 1) learning spatially varying pooling fields that improve model performance without the extra computation cost associated with multi-scale pooling, and 2) learning a dynamic computation policy for each pixel to decrease total computation while maintaining accuracy. We extensively evaluate PAG on a variety of per-pixel labeling tasks, including semantic segmentation, boundary detection, monocular depth and surface normal estimation. We demonstrate that PAG allows competitive or state-of-the-art performance on these tasks. Our experiments show that PAG learns dynamic spatial allocation of computation over the input image which provides better performance trade-offs compared to related approaches (e.g., truncating deep models or dynamically skipping whole layers). Generally, we observe PAG can reduce computation by 10% without noticeable loss in accuracy and performance degrades gracefully when imposing stronger computational constraints.