ReGLA: Efficient Receptive-Field Modeling with Gated Linear Attention Network

Balancing accuracy and latency on high-resolution images is a critical challenge for lightweight models, particularly for Transformer-based architectures that often suffer from excessive latency. To address this issue, we introduce ReGLA, a series of lightweight hybrid networks, which integrates efficient convolutions for local feature extraction with ReLU-based gated linear attention for global modeling. The design incorporates three key innovations: the Efficient Large Receptive Field (ELRF) module for enhancing convolutional efficiency while preserving a large receptive field; the ReLU Gated Modulated Attention (RGMA) module for maintaining linear complexity while enhancing local feature representation; and a multi-teacher distillation strategy to boost performance on downstream tasks. Extensive experiments validate the superiority of ReGLA; particularly the ReGLA-M achieves 80.85% Top-1 accuracy on ImageNet-1K at 224px, with only 4.98 ms latency at 512px. Furthermore, ReGLA outperforms similarly scaled iFormer models in downstream tasks, achieving gains of 3.1% AP on COCO object detection and 3.6% mIoU on ADE20K semantic segmentation, establishing it as a state-of-the-art solution for high-resolution visual applications.