# ParC-Net: Position Aware Circular Convolution with Merits from ConvNets and Transformer

arXiv: https://arxiv.org/abs/2203.03952 · HTML: https://mlsanity.com/paper/2203.03952/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 41 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer](https://mlsanity.com/paper/2110.02178/) (2021)
- [CoAtNet: Marrying Convolution and Attention for All Data Sizes](https://mlsanity.com/paper/2106.04803/) (2021)
- [MetaFormer Is Actually What You Need for Vision](https://mlsanity.com/paper/2111.11418/) (2021)
- [ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases](https://mlsanity.com/paper/2103.10697/) (2021)
- [LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference](https://mlsanity.com/paper/2104.01136/) (2021)
- [GhostNet: More Features from Cheap Operations](https://mlsanity.com/paper/1911.11907/) (2019)
- [Early Convolutions Help Transformers See Better](https://mlsanity.com/paper/2106.14881/) (2021)
- [Bottleneck Transformers for Visual Recognition](https://mlsanity.com/paper/2101.11605/) (2021)

## Built on by (most cited first, 28 of 28)

- [Spike-driven Transformer](https://mlsanity.com/paper/2307.01694/) (2023)
- [RepViT: Revisiting Mobile CNN From ViT Perspective](https://mlsanity.com/paper/2307.09283/) (2023)
- [SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications](https://mlsanity.com/paper/2303.15446/) (2023)
- [Omni Aggregation Networks for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2304.10244/) (2023)
- [Lightweight Vision Transformer with Bidirectional Interaction](https://mlsanity.com/paper/2306.00396/) (2023)
- [TransXNet: Learning Both Global and Local Dynamics with a Dual Dynamic Token Mixer for Visual Recognition](https://mlsanity.com/paper/2310.19380/) (2023)
- [RGB no more: Minimally-decoded JPEG Vision Transformers](https://mlsanity.com/paper/2211.16421/) (2022)
- [Lightweight Pixel Difference Networks for Efficient Visual Representation Learning](https://mlsanity.com/paper/2402.00422/) (2024)
- [High-Order Structure Based Middle-Feature Learning for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2312.07853/) (2023)
- [Efficient Modulation for Vision Networks](https://mlsanity.com/paper/2403.19963/) (2024)
- [LSNet: See Large, Focus Small](https://mlsanity.com/paper/2503.23135/) (2025)
- [Fcaformer: Forward Cross Attention in Hybrid Vision Transformer](https://mlsanity.com/paper/2211.07198/) (2022)
- [NAR-Former V2: Rethinking Transformer for Universal Neural Network Representation Learning](https://mlsanity.com/paper/2306.10792/) (2023)
- [DPNet: Dual-Path Network for Real-time Object Detection with Lightweight Attention](https://mlsanity.com/paper/2209.13933/) (2022)
- [M3FAS: An Accurate and Robust MultiModal Mobile Face Anti-Spoofing System](https://mlsanity.com/paper/2301.12831/) (2023)
- [Rethinking Local Perception in Lightweight Vision Transformer](https://mlsanity.com/paper/2303.17803/) (2023)
- [From Tiny Machine Learning to Tiny Deep Learning: A Survey](https://mlsanity.com/paper/2506.18927/) (2025)
- [ParCNetV2: Oversized Kernel with Enhanced Attention](https://mlsanity.com/paper/2211.07157/) (2022)
- [RepNeXt: A Fast Multi-Scale CNN using Structural Reparameterization](https://mlsanity.com/paper/2406.16004/) (2024)
- [EffLoc: Lightweight Vision Transformer for Efficient 6-DOF Camera Relocalization](https://mlsanity.com/paper/2402.13537/) (2024)
- [YOLO-Ant: A Lightweight Detector via Depthwise Separable Convolutional and Large Kernel Design for Antenna Interference Source Detection](https://mlsanity.com/paper/2402.12641/) (2024)
- [3D-RCNet: Learning from Transformer to Build a 3D Relational ConvNet for Hyperspectral Image Classification](https://mlsanity.com/paper/2408.13728/) (2024)
- [DA-Occ: Direction-Aware 2D Convolution for Efficient and Geometry-Preserving 3D Occupancy Prediction in Autonomous Driving](https://mlsanity.com/paper/2507.23599/) (2025)
- [Fast-ParC: Capturing Position Aware Global Feature for ConvNets and ViTs](https://mlsanity.com/paper/2210.04020/) (2022)
- [Boosting Binary Neural Networks via Dynamic Thresholds Learning](https://mlsanity.com/paper/2211.02292/) (2022)
- [Star-Net: Improving Single Image Desnowing Model With More Efficient Connection and Diverse Feature Interaction](https://mlsanity.com/paper/2303.09988/) (2023)
- [DMON: A Simple yet Effective Approach for Argument Structure Learning](https://mlsanity.com/paper/2405.01216/) (2024)
- [RecConv: Efficient Recursive Convolutions for Multi-Frequency Representations](https://mlsanity.com/paper/2412.19628/) (2024)

## Abstract (arXiv)

Recently, vision transformers started to show impressive results which outperform large convolution based models significantly. However, in the area of small models for mobile or resource constrained devices, ConvNet still has its own advantages in both performance and model complexity. We propose ParC-Net, a pure ConvNet based backbone model that further strengthens these advantages by fusing the merits of vision transformers into ConvNets. Specifically, we propose position aware circular convolution (ParC), a light-weight convolution op which boasts a global receptive field while producing location sensitive features as in local convolutions. We combine the ParCs and squeeze-exictation ops to form a meta-former like model block, which further has the attention mechanism like transformers. The aforementioned block can be used in plug-and-play manner to replace relevant blocks in ConvNets or transformers. Experiment results show that the proposed ParC-Net achieves better performance than popular light-weight ConvNets and vision transformer based models in common vision tasks and datasets, while having fewer parameters and faster inference speed. For classification on ImageNet-1k, ParC-Net achieves 78.6% top-1 accuracy with about 5.0 million parameters, saving 11% parameters and 13% computational cost but gaining 0.2% higher accuracy and 23% faster inference speed (on ARM based Rockchip RK3288) compared with MobileViT, and uses only 0.5 times parameters but gaining 2.7% accuracy compared with DeIT. On MS-COCO object detection and PASCAL VOC segmentation tasks, ParC-Net also shows better performance. Source code is available at https://github.com/hkzhang91/ParC-Net