# Linear Context Transform Block

arXiv: https://arxiv.org/abs/1909.03834 · HTML: https://mlsanity.com/paper/1909.03834/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2019.

## Built on by (most cited first, 5 of 5)

- [Augmenting Deep Classifiers with Polynomial Neural Networks](https://mlsanity.com/paper/2104.07916/) (2021)
- [CSA-Net: Channel-wise Spatially Autocorrelated Attention Networks](https://mlsanity.com/paper/2405.05755/) (2024)
- [Context-LGM: Leveraging Object-Context Relation for Context-Aware Object Recognition](https://mlsanity.com/paper/2110.04042/) (2021)
- [Parameter-Free Bio-Inspired Channel Attention for Enhanced Cardiac MRI Reconstruction](https://mlsanity.com/paper/2505.23872/) (2025)
- [Pathology Context Recalibration Network for Ocular Disease Recognition](https://mlsanity.com/paper/2512.24066/) (2025)

## Abstract (arXiv)

Squeeze-and-Excitation (SE) block presents a channel attention mechanism for modeling global context via explicitly capturing dependencies across channels. However, we are still far from understanding how the SE block works. In this work, we first revisit the SE block, and then present a detailed empirical study of the relationship between global context and attention distribution, based on which we propose a simple yet effective module, called Linear Context Transform (LCT) block. We divide all channels into different groups and normalize the globally aggregated context features within each channel group, reducing the disturbance from irrelevant channels. Through linear transform of the normalized context features, we model global context for each channel independently. The LCT block is extremely lightweight and easy to be plugged into different backbone models while with negligible parameters and computational burden increase. Extensive experiments show that the LCT block outperforms the SE block in image classification task on the ImageNet and object detection/segmentation on the COCO dataset with different backbone models. Moreover, LCT yields consistent performance gains over existing state-of-the-art detection architectures, e.g., 1.5~1.7% APbbox and 1.0~1.2% APmask improvements on the COCO benchmark, irrespective of different baseline models of varied capacities. We hope our simple yet effective approach will shed some light on future research of attention-based models.