# Pixel-Adaptive Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1904.05373 · HTML: https://mlsanity.com/paper/1904.05373/

## Key facts

- Cited by 126 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2019.

## Built on by (most cited first, 30 of 126)

- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [Focal Sparse Convolutional Networks for 3D Object Detection](https://mlsanity.com/paper/2204.12463/) (2022)
- [Learning Affinity from Attention: End-to-End Weakly-Supervised Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.02664/) (2022)
- [A Survey of Visual Transformers](https://mlsanity.com/paper/2111.06091/) (2021)
- [FeatUp: A Model-Agnostic Framework for Features at Any Resolution](https://mlsanity.com/paper/2403.10516/) (2024)
- [Involution: Inverting the Inherence of Convolution for Visual Recognition](https://mlsanity.com/paper/2103.06255/) (2021)
- [Iterative Filter Adaptive Network for Single Image Defocus Deblurring](https://mlsanity.com/paper/2108.13610/) (2021)
- [Discrete Cosine Transform Network for Guided Depth Map Super-Resolution](https://mlsanity.com/paper/2104.06977/) (2021)
- [Exploring Relational Context for Multi-Task Dense Prediction](https://mlsanity.com/paper/2104.13874/) (2021)
- [Dynamic Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2202.09769/) (2022)
- [Towards Fast and Accurate Real-World Depth Super-Resolution: Benchmark Dataset and Baseline](https://mlsanity.com/paper/2104.06174/) (2021)
- [OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving](https://mlsanity.com/paper/2102.07448/) (2021)
- [Fine-grained Semantics-aware Representation Enhancement for Self-supervised Monocular Depth Estimation](https://mlsanity.com/paper/2108.08829/) (2021)
- [Decoupled Dynamic Filter Networks](https://mlsanity.com/paper/2104.14107/) (2021)
- [A Dynamic Multi-Scale Voxel Flow Network for Video Prediction](https://mlsanity.com/paper/2303.09875/) (2023)
- [Adaptive Rotated Convolution for Rotated Object Detection](https://mlsanity.com/paper/2303.07820/) (2023)
- [Self-Supervised Material and Texture Representation Learning for Remote Sensing Tasks](https://mlsanity.com/paper/2112.01715/) (2021)
- [Adaptive Surface Normal Constraint for Depth Estimation](https://mlsanity.com/paper/2103.15483/) (2021)
- [Enhancing Visibility in Nighttime Haze Images Using Guided APSF and Gradient Adaptive Convolution](https://mlsanity.com/paper/2308.01738/) (2023)
- [HyperE2VID: Improving Event-Based Video Reconstruction via Hypernetworks](https://mlsanity.com/paper/2305.06382/) (2023)
- [DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalization](https://mlsanity.com/paper/2209.05227/) (2022)
- [Frequency-Adaptive Dilated Convolution for Semantic Segmentation](https://mlsanity.com/paper/2403.05369/) (2024)
- [SOLO: A Simple Framework for Instance Segmentation](https://mlsanity.com/paper/2106.15947/) (2021)
- [Joint Implicit Image Function for Guided Depth Super-Resolution](https://mlsanity.com/paper/2107.08717/) (2021)
- [BridgeNet: A Joint Learning Network of Depth Map Super-Resolution and Monocular Depth Estimation](https://mlsanity.com/paper/2107.12541/) (2021)
- [GaitGCI: Generative Counterfactual Intervention for Gait Recognition](https://mlsanity.com/paper/2306.03428/) (2023)
- [Learning Scene Structure Guidance via Cross-Task Knowledge Transfer for Single Depth Super-Resolution](https://mlsanity.com/paper/2103.12955/) (2021)
- [PointConvFormer: Revenge of the Point-based Convolution](https://mlsanity.com/paper/2208.02879/) (2022)
- [Spherical Space Feature Decomposition for Guided Depth Map Super-Resolution](https://mlsanity.com/paper/2303.08942/) (2023)
- [Learning Graph Regularisation for Guided Super-Resolution](https://mlsanity.com/paper/2203.14297/) (2022)

## Abstract (arXiv)

Convolutions are the fundamental building block of CNNs. The fact that their weights are spatially shared is one of the main reasons for their widespread use, but it also is a major limitation, as it makes convolutions content agnostic. We propose a pixel-adaptive convolution (PAC) operation, a simple yet effective modification of standard convolutions, in which the filter weights are multiplied with a spatially-varying kernel that depends on learnable, local pixel features. PAC is a generalization of several popular filtering techniques and thus can be used for a wide range of use cases. Specifically, we demonstrate state-of-the-art performance when PAC is used for deep joint image upsampling. PAC also offers an effective alternative to fully-connected CRF (Full-CRF), called PAC-CRF, which performs competitively, while being considerably faster. In addition, we also demonstrate that PAC can be used as a drop-in replacement for convolution layers in pre-trained networks, resulting in consistent performance improvements.