# CNNtention: Can CNNs do better with Attention?

arXiv: https://arxiv.org/abs/2412.11657 · HTML: https://mlsanity.com/paper/2412.11657/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [CA-Net: Comprehensive Attention Convolutional Neural Networks for Explainable Medical Image Segmentation](https://mlsanity.com/paper/2009.10549/) (2020)
- [NoisyNN: Exploring the Impact of Information Entropy Change in Learning Systems](https://mlsanity.com/paper/2309.10625/) (2023)
- [Weighted Residuals for Very Deep Networks](https://mlsanity.com/paper/1605.08831/) (2016)
- [Contextual Convolutional Neural Networks](https://mlsanity.com/paper/2108.07387/) (2021)
- [CSANet: Channel Spatial Attention Network for Robust 3D Face Alignment and Reconstruction](https://mlsanity.com/paper/2405.19659/) (2024)

## Abstract (arXiv)

Convolutional Neural Networks (CNNs) have been the standard for image classification tasks for a long time, but more recently attention-based mechanisms have gained traction. This project aims to compare traditional CNNs with attention-augmented CNNs across an image classification task. By evaluating and comparing their performance, accuracy and computational efficiency, the project will highlight benefits and trade-off of the localized feature extraction of traditional CNNs and the global context capture in attention-augmented CNNs. By doing this, we can reveal further insights into their respective strengths and weaknesses, guide the selection of models based on specific application needs and ultimately, enhance understanding of these architectures in the deep learning community. This was our final project for CS7643 Deep Learning course at Georgia Tech.