mlsanityData to 7 Oct 2026

Augmenting Convolutional networks with attention-based aggregation

arXiv 2112.13692 ↗First posted 27 Dec 2021cs.CVcited by 29

Not on the map. Why?

Abstract (arXiv)

We show how to augment any convolutional network with an attention-based global map to achieve non-local reasoning. We replace the final average pooling by an attention-based aggregation layer akin to a single transformer block, that weights how the patches are involved in the classification decision. We plug this learned aggregation layer with a simplistic patch-based convolutional network parametrized by 2 parameters (width and depth). In contrast with a pyramidal design, this architecture family maintains the input patch resolution across all the layers. It yields surprisingly competitive trade-offs between accuracy and complexity, in particular in terms of memory consumption, as shown by our experiments on various computer vision tasks: object classification, image segmentation and detection.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 64 of its 78 references that are in the map. First posted on arXiv 27 Dec 2021.

Builds on

64 of 78 references.

References · 64

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2017Mask R-CNNcited by 7,259
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  8. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  9. 2016Layer Normalizationcited by 5,579
  10. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  11. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  12. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
38 more
  1. 2017Squeeze-and-Excitation Networkscited by 4,763
  2. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  3. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  4. 2016Wide Residual Networkscited by 3,057
  5. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  6. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  7. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  8. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  9. 2017Non-local Neural Networkscited by 2,290
  10. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  11. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  12. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  13. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  14. 2017Decoupled Weight Decay Regularizationcited by 1,722
  15. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  16. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  17. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  18. 2016Deep Networks with Stochastic Depthcited by 973
  19. 2020Designing Network Design Spacescited by 954
  20. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 936
  21. 2021Going deeper with Image Transformerscited by 759
  22. 2019Large Batch Optimization for Deep Learning: Training BERT in 76 minutescited by 671
  23. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  24. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  25. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 575
  26. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  27. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  28. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 526
  29. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  30. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  31. 2021Early Convolutions Help Transformers See Bettercited by 395
  32. 2021ResNet strikes back: An improved training procedure in timmcited by 365
  33. 2019Attention Augmented Convolutional Networkscited by 363
  34. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
  35. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 359
  36. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
  37. 2021XCiT: Cross-Covariance Image Transformerscited by 335
  38. 2020Visual Transformers: Token-based Image Representation and Processing for Computer Visioncited by 333

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 4; 2024: 10; 2025: 3; 2026: 2.

29 papers.

Sort:

By area:

  1. 2022DeiT III: Revenge of the ViTcited by 417
  2. 2022Focal Modulation Networkscited by 186
  3. 2022Three things everyone should know about Vision Transformerscited by 90
  4. 2022Adversarial Masking for Self-Supervised Learningcited by 78
  5. 2022MogaNet: Multi-order Gated Aggregation Networkcited by 59
  6. 2024Rethinking Patch Dependence for Masked Autoencoderscited by 29
  7. 2022Peripheral Vision Transformercited by 27
  8. 2022Backdoor Attacks on Vision Transformerscited by 13
  9. 2022Continual Transformers: Redundancy-Free Attention for Online Inferencecited by 12
  10. 2024DenseNets Reloaded: Paradigm Shift Beyond ResNets and ViTscited by 12
  11. 2023PELA: Learning Parameter-Efficient Models with Low-Rank Approximationcited by 9
  12. 2023Astroformer: More Data Might not be all you need for Classificationcited by 6
17 more
  1. 2023A vector quantized masked autoencoder for audiovisual speech emotion recognitioncited by 6
  2. 2023Attention-Map Augmentation for Hypercomplex Breast Cancer Classificationcited by 6
  3. 2026From Pixels to Patches: Pooling Strategies for Earth Embeddingscited by 4
  4. 2022A Close Look at Spatial Modeling: From Attention to Convolutioncited by 3
  5. 2024On Train-Test Class Overlap and Detection for Image Retrievalcited by 3
  6. 2024Efficient generative adversarial networks using linear additive-attention Transformerscited by 2
  7. 2024HyenaPixel: Global Image Context with Convolutionscited by 2
  8. 2024CA-Stream: Attention-based pooling for interpretable image recognitioncited by 2
  9. 2024Scaling Structure Aware Virtual Screening to Billions of Molecules with SPRINTcited by 2
  10. 2025Spline-based Transformerscited by 2
  11. 2022Deep Superpixel Generation and Clustering for Weakly Supervised Segmentation of Brain Tumors in MR Imagescited by 1
  12. 2024A new method for optical steel rope non-destructive damage detectioncited by 0
  13. 2024A Survey on Cell Nuclei Instance Segmentation and Classification: Leveraging Context and Attentioncited by 0
  14. 2024SleepNet and DreamNet: Enriching and Reconstructing Representations for Consolidated Visual Classificationcited by 0
  15. 2025ACE: A Cardinality Estimator for Set-Valued Queriescited by 0
  16. 2025Structure-Aware Contrastive Learning with Fine-Grained Binding Representations for Drug Discoverycited by 0
  17. 2026Scalable Music Cover Retrieval Using Lyrics-Aligned Audio Embeddingscited by 0