# GPCA: A Probabilistic Framework for Gaussian Process Embedded Channel Attention

arXiv: https://arxiv.org/abs/2003.04575 · HTML: https://mlsanity.com/paper/2003.04575/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2020.

## Built on by (most cited first, 7 of 7)

- [ViTAS: Vision Transformer Architecture Search](https://mlsanity.com/paper/2106.13700/) (2021)
- [Domain Generalization via Frequency-domain-based Feature Disentanglement and Interaction](https://mlsanity.com/paper/2201.08029/) (2022)
- [Probabilistic Attention based on Gaussian Processes for Deep Multiple Instance Learning](https://mlsanity.com/paper/2302.04061/) (2023)
- [A Generic Shared Attention Mechanism for Various Backbone Neural Networks](https://mlsanity.com/paper/2210.16101/) (2022)
- [Deep Class Incremental Learning from Decentralized Data](https://mlsanity.com/paper/2203.05984/) (2022)
- [ScaleNet: Searching for the Model to Scale](https://mlsanity.com/paper/2207.07267/) (2022)
- [Density Adaptive Attention is All You Need: Robust Parameter-Efficient Fine-Tuning Across Multiple Modalities](https://mlsanity.com/paper/2401.11143/) (2024)

## Abstract (arXiv)

Channel attention mechanisms have been commonly applied in many visual tasks for effective performance improvement. It is able to reinforce the informative channels as well as to suppress the useless channels. Recently, different channel attention modules have been proposed and implemented in various ways. Generally speaking, they are mainly based on convolution and pooling operations. In this paper, we propose Gaussian process embedded channel attention (GPCA) module and further interpret the channel attention schemes in a probabilistic way. The GPCA module intends to model the correlations among the channels, which are assumed to be captured by beta distributed variables. As the beta distribution cannot be integrated into the end-to-end training of convolutional neural networks (CNNs) with a mathematically tractable solution, we utilize an approximation of the beta distribution to solve this problem. To specify, we adapt a Sigmoid-Gaussian approximation, in which the Gaussian distributed variables are transferred into the interval [0,1]. The Gaussian process is then utilized to model the correlations among different channels. In this case, a mathematically tractable solution is derived. The GPCA module can be efficiently implemented and integrated into the end-to-end training of the CNNs. Experimental results demonstrate the promising performance of the proposed GPCA module. Codes are available at https://github.com/PRIS-CV/GPCA.