mlsanityData to 7 Oct 2026

UPANets: Learning from the Universal Pixel Attention Networks

arXiv 2103.08640 ↗First posted 15 Mar 2021cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Among image classification, skip and densely-connection-based networks have dominated most leaderboards. Recently, from the successful development of multi-head attention in natural language processing, it is sure that now is a time of either using a Transformer-like model or hybrid CNNs with attention. However, the former need a tremendous resource to train, and the latter is in the perfect balance in this direction. In this work, to make CNNs handle global and local information, we proposed UPANets, which equips channel-wise attention with a hybrid skip-densely-connection structure. Also, the extreme-connection structure makes UPANets robust with a smoother loss landscape. In experiments, UPANets surpassed most well-known and widely-used SOTAs with an accuracy of 96.47% in Cifar-10, 80.29% in Cifar-100, and 67.67% in Tiny Imagenet. Most importantly, these performances have high parameters efficiency and only trained in one customer-based GPU. We share implementing code of UPANets in https://github.com/hanktseng131415go/UPANets.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 37 of its 44 references that are in the map. First posted on arXiv 15 Mar 2021.

Builds on

37 of 44 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2016Densely Connected Convolutional Networkscited by 6,691
  6. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2017Squeeze-and-Excitation Networkscited by 4,763
  9. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  10. 2014Going Deeper with Convolutionscited by 4,231
  11. 2016Wide Residual Networkscited by 3,057
  12. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
25 more
  1. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
  2. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  3. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  4. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  5. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  6. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  7. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  8. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  9. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  10. 2021Transformer in Transformercited by 632
  11. 2019Res2Net: A New Multi-scale Backbone Architecturecited by 468
  12. 2017Deep Layer Aggregationcited by 436
  13. 2019GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyondcited by 331
  14. 2020Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixupcited by 260
  15. 2016Deep Pyramidal Residual Networkscited by 163
  16. 2021Generative Adversarial Transformerscited by 120
  17. 2017Dual Path Networkscited by 119
  18. 2020TResNet: High Performance GPU-Dedicated Architecturecited by 119
  19. 2014Salient Object Detection: A Surveycited by 98
  20. 2020Each Part Matters: Local Patterns Facilitate Cross-view Geo-localizationcited by 94
  21. 2019SRM : A Style-based Recalibration Module for Convolutional Neural Networkscited by 65
  22. 2020SpinalNet: Deep Neural Network with Gradual Inputcited by 40
  23. 2020Non-convex Learning via Replica Exchange Stochastic Gradient MCMCcited by 33
  24. 2019DenseNet Models for Tiny ImageNet Classificationcited by 17
  25. 2016Deep Residual Networks with Exponential Linear Unitcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 0; 2023: 3; 2024: 0; 2025: 0; 2026: 1.

7 papers.

  1. 2021Efficient Sharpness-aware Minimization for Improved Training of Neural Networkscited by 137
  2. 2021Perturbated Gradients Updating within Unit Space for Deep Learningcited by 3
  3. 2021Full-attention based Neural Architecture Search using Context Auto-regressioncited by 2
  4. 2023Real-time Automatic M-mode Echocardiography Measurement with Panel Attention from Local-to-Global Pixelscited by 1
  5. 2023Leader-Follower Neural Networks with Local Error Signals Inspired by Complex Collectivescited by 1
  6. 2026OptiRoulette Optimizer: A New Stochastic Meta-Optimizer for up to 5.3x Faster Convergencecited by 1
  7. 2023GoogLe2Net: Going Transverse with Convolutionscited by 0