# GhostNetV2: Enhance Cheap Operation with Long-Range Attention

arXiv: https://arxiv.org/abs/2211.12905 · HTML: https://mlsanity.com/paper/2211.12905/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 57 references that are in the map.
- First posted on arXiv 23 Nov 2022.

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer](https://mlsanity.com/paper/2110.02178/) (2021)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [GhostNet: More Features from Cheap Operations](https://mlsanity.com/paper/1911.11907/) (2019)
- [Speeding up Convolutional Neural Networks with Low Rank Expansions](https://mlsanity.com/paper/1405.3866/) (2014)
- [Coordinate Attention for Efficient Mobile Network Design](https://mlsanity.com/paper/2103.02907/) (2021)

## Built on by (most cited first, 30 of 44)

- [MobileNetV4 -- Universal Models for the Mobile Ecosystem](https://mlsanity.com/paper/2404.10518/) (2024)
- [Poly Kernel Inception Network for Remote Sensing Detection](https://mlsanity.com/paper/2403.06258/) (2024)
- [CascadedGaze: Efficiency in Global Context Extraction for Image Restoration](https://mlsanity.com/paper/2401.15235/) (2024)
- [Efficient Modulation for Vision Networks](https://mlsanity.com/paper/2403.19963/) (2024)
- [MobileMamba: Lightweight Multi-Receptive Visual Mamba Network](https://mlsanity.com/paper/2411.15941/) (2024)
- [Computation-efficient Deep Learning for Computer Vision: A Survey](https://mlsanity.com/paper/2308.13998/) (2023)
- [GhostNetV3: Exploring the Training Strategies for Compact Models](https://mlsanity.com/paper/2404.11202/) (2024)
- [MobileIE: An Extremely Lightweight and Effective ConvNet for Real-Time Image Enhancement on Mobile Devices](https://mlsanity.com/paper/2507.01838/) (2025)
- [iFormer: Integrating ConvNet and Transformer for Mobile Application](https://mlsanity.com/paper/2501.15369/) (2025)
- [FalconNet: Factorization for the Light-weight ConvNets](https://mlsanity.com/paper/2306.06365/) (2023)
- [All-optical image denoising using a diffractive visual processor](https://mlsanity.com/paper/2309.09215/) (2023)
- [ChangeViT: Unleashing Plain Vision Transformers for Change Detection](https://mlsanity.com/paper/2406.12847/) (2024)
- [LowFormer: Hardware Efficient Design for Convolutional Transformer Backbones](https://mlsanity.com/paper/2409.03460/) (2024)
- [AIM 2024 Challenge on UHD Blind Photo Quality Assessment](https://mlsanity.com/paper/2409.16271/) (2024)
- [COSMIC: Compress Satellite Images Efficiently via Diffusion Compensation](https://mlsanity.com/paper/2410.01698/) (2024)
- [LMFCA-Net: A Lightweight Model for Multi-Channel Speech Enhancement with Efficient Narrow-Band and Cross-Band Attention](https://mlsanity.com/paper/2502.11462/) (2025)
- [Towards Ancient Plant Seed Classification: A Benchmark Dataset and Baseline Model](https://mlsanity.com/paper/2512.18247/) (2025)
- [CMIP X-MOS: Improving Climate Models with Extreme Model Output Statistics](https://mlsanity.com/paper/2311.03370/) (2023)
- [Agricultural Object Detection with You Look Only Once (YOLO) Algorithm: A Bibliometric and Systematic Literature Review](https://mlsanity.com/paper/2401.10379/) (2024)
- [Towards Fundamentally Scalable Model Selection: Asymptotically Fast Update and Selection](https://mlsanity.com/paper/2406.07536/) (2024)
- [LeYOLO, New Embedded Architecture for Object Detection](https://mlsanity.com/paper/2406.14239/) (2024)
- [Towards Real-time Video Compressive Sensing on Mobile Devices](https://mlsanity.com/paper/2408.07530/) (2024)
- [Mobile-Friendly Deep Learning for Plant Disease Detection: A Lightweight CNN Benchmark Across 101 Classes of 33 Crops](https://mlsanity.com/paper/2508.10817/) (2025)
- [Attention-Driven Lightweight Model for Pigmented Skin Lesion Detection](https://mlsanity.com/paper/2308.02119/) (2023)
- [Better YOLO with Attention-Augmented Network and Enhanced Generalization Performance for Safety Helmet Detection](https://mlsanity.com/paper/2405.02591/) (2024)
- [Multiple Prior Representation Learning for Self-Supervised Monocular Depth Estimation via Hybrid Transformer](https://mlsanity.com/paper/2406.08928/) (2024)
- [Efficient Oriented Object Detection with Enhanced Small Object Recognition in Aerial Images](https://mlsanity.com/paper/2412.12562/) (2024)
- [MSConv: Multiplicative and Subtractive Convolution for Face Recognition](https://mlsanity.com/paper/2503.06187/) (2025)
- [Fraesormer: Learning Adaptive Sparse Transformer for Efficient Food Recognition](https://mlsanity.com/paper/2503.11995/) (2025)
- [Expressive yet Efficient Feature Expansion with Adaptive Cross-Hadamard Products](https://mlsanity.com/paper/2505.22226/) (2025)

## Abstract (arXiv)

Light-weight convolutional neural networks (CNNs) are specially designed for applications on mobile devices with faster inference speed. The convolutional operation can only capture local information in a window region, which prevents performance from being further improved. Introducing self-attention into convolution can capture global information well, but it will largely encumber the actual speed. In this paper, we propose a hardware-friendly attention mechanism (dubbed DFC attention) and then present a new GhostNetV2 architecture for mobile applications. The proposed DFC attention is constructed based on fully-connected layers, which can not only execute fast on common hardware but also capture the dependence between long-range pixels. We further revisit the expressiveness bottleneck in previous GhostNet and propose to enhance expanded features produced by cheap operations with DFC attention, so that a GhostNetV2 block can aggregate local and long-range information simultaneously. Extensive experiments demonstrate the superiority of GhostNetV2 over existing architectures. For example, it achieves 75.3% top-1 accuracy on ImageNet with 167M FLOPs, significantly suppressing GhostNetV1 (74.5%) with a similar computational cost. The source code will be available at https://github.com/huawei-noah/Efficient-AI-Backbones/tree/master/ghostnetv2_pytorch and https://gitee.com/mindspore/models/tree/master/research/cv/ghostnetv2.