# Context-aware Attentional Pooling (CAP) for Fine-grained Visual Classification

arXiv: https://arxiv.org/abs/2101.06635 · HTML: https://mlsanity.com/paper/2101.06635/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 68 references that are in the map.
- First posted on arXiv 17 Jan 2021.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [Deep Layer Aggregation](https://mlsanity.com/paper/1707.06484/) (2017)
- [Rethinking ImageNet Pre-training](https://mlsanity.com/paper/1811.08883/) (2018)
- [Attention Augmented Convolutional Networks](https://mlsanity.com/paper/1904.09925/) (2019)
- [Exploring Self-attention for Image Recognition](https://mlsanity.com/paper/2004.13621/) (2020)
- [Attention on Attention for Image Captioning](https://mlsanity.com/paper/1908.06954/) (2019)
- [Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning](https://mlsanity.com/paper/1806.06193/) (2018)
- [Image Captioning: Transforming Objects into Words](https://mlsanity.com/paper/1906.05963/) (2019)
- [Learning to Navigate for Fine-grained Classification](https://mlsanity.com/paper/1809.00287/) (2018)
- [Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognition](https://mlsanity.com/paper/1903.06150/) (2019)
- [The Application of Two-level Attention Models in Deep Convolutional Neural Network for Fine-grained Image Classification](https://mlsanity.com/paper/1411.6447/) (2014)
- [Weakly Supervised Complementary Parts Models for Fine-Grained Image Classification from the Bottom Up](https://mlsanity.com/paper/1903.02827/) (2019)
- [Towards Faster Training of Global Covariance Pooling Networks by Iterative Matrix Square Root Normalization](https://mlsanity.com/paper/1712.01034/) (2017)
- [Maximum-Entropy Fine-Grained Classification](https://mlsanity.com/paper/1809.05934/) (2018)
- [The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification](https://mlsanity.com/paper/2002.04264/) (2020)
- [Cross-X Learning for Fine-Grained Visual Categorization](https://mlsanity.com/paper/1909.04412/) (2019)
- [Learnable pooling with Context Gating for video classification](https://mlsanity.com/paper/1706.06905/) (2017)
- [Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition](https://mlsanity.com/paper/1806.05372/) (2018)
- [See Better Before Looking Closer: Weakly Supervised Data Augmentation Network for Fine-Grained Visual Classification](https://mlsanity.com/paper/1901.09891/) (2019)
- [Object-Part Attention Model for Fine-grained Image Classification](https://mlsanity.com/paper/1704.01740/) (2017)

## Built on by (most cited first, 30 of 31)

- [MetaFormer: A Unified Meta Framework for Fine-Grained Recognition](https://mlsanity.com/paper/2203.02751/) (2022)
- [Fine-grained Visual Classification with High-temperature Refinement and Background Suppression](https://mlsanity.com/paper/2303.06442/) (2023)
- [SR-GNN: Spatial Relation-aware Graph Neural Network for Fine-Grained Image Categorization](https://mlsanity.com/paper/2209.02109/) (2022)
- [Dynamic MLP for Fine-Grained Image Classification by Leveraging Geographical and Temporal Information](https://mlsanity.com/paper/2203.03253/) (2022)
- [Open-TransMind: A New Baseline and Benchmark for 1st Foundation Model Challenge of Intelligent Transportation](https://mlsanity.com/paper/2304.06051/) (2023)
- [A Novel Plug-in Module for Fine-Grained Visual Classification](https://mlsanity.com/paper/2202.03822/) (2022)
- [Self-Attentive Pooling for Efficient Deep Learning](https://mlsanity.com/paper/2209.07659/) (2022)
- [Switchable Self-attention Module](https://mlsanity.com/paper/2209.05680/) (2022)
- [Global-Local Similarity for Efficient Fine-Grained Image Recognition with Vision Transformers](https://mlsanity.com/paper/2407.12891/) (2024)
- [A Feature Consistency Driven Attention Erasing Network for Fine-Grained Image Retrieval](https://mlsanity.com/paper/2110.04479/) (2021)
- [Salient Mask-Guided Vision Transformer for Fine-Grained Classification](https://mlsanity.com/paper/2305.07102/) (2023)
- [A Novel Image Classification Framework Based on Variational Quantum Algorithms](https://mlsanity.com/paper/2312.07932/) (2023)
- [Context-Semantic Quality Awareness Network for Fine-Grained Visual Categorization](https://mlsanity.com/paper/2403.10298/) (2024)
- [M2Former: Multi-Scale Patch Selection for Fine-Grained Visual Recognition](https://mlsanity.com/paper/2308.02161/) (2023)
- [Learning Time in Static Classifiers](https://mlsanity.com/paper/2511.12321/) (2025)
- [Relational Proxies: Emergent Relationships as Fine-Grained Discriminators](https://mlsanity.com/paper/2210.02149/) (2022)
- [Deep Neural Networks Fused with Textures for Image Classification](https://mlsanity.com/paper/2308.01813/) (2023)
- [Cross-Level Multi-Instance Distillation for Self-Supervised Fine-Grained Visual Categorization](https://mlsanity.com/paper/2401.08860/) (2024)
- [ConceptHash: Interpretable Fine-Grained Hashing via Concept Discovery](https://mlsanity.com/paper/2406.08457/) (2024)
- [Sequential Random Network for Fine-grained Image Classification](https://mlsanity.com/paper/2103.07230/) (2021)
- [An attention-driven hierarchical multi-scale representation for visual recognition](https://mlsanity.com/paper/2110.12178/) (2021)
- [Explored An Effective Methodology for Fine-Grained Snake Recognition](https://mlsanity.com/paper/2207.11637/) (2022)
- [Fine-grained Classification of Solder Joints with α-skew Jensen-Shannon Divergence](https://mlsanity.com/paper/2209.09857/) (2022)
- [Semantic Feature Integration network for Fine-grained Visual Classification](https://mlsanity.com/paper/2302.10275/) (2023)
- [ELFIS: Expert Learning for Fine-grained Image Recognition Using Subsets](https://mlsanity.com/paper/2303.09269/) (2023)
- [Transitivity Recovering Decompositions: Interpretable and Robust Fine-Grained Relationships](https://mlsanity.com/paper/2310.15999/) (2023)
- [Hybrid Feature Collaborative Reconstruction Network for Few-Shot Fine-Grained Image Classification](https://mlsanity.com/paper/2407.02123/) (2024)
- [Navigating limitations with precision: A fine-grained ensemble approach to wrist pathology recognition on a limited x-ray dataset](https://mlsanity.com/paper/2412.13884/) (2024)
- [A Review on Coarse to Fine-Grained Animal Action Recognition](https://mlsanity.com/paper/2506.01214/) (2025)
- [Women Sport Actions Dataset for Visual Classification Using Small Scale Training Data](https://mlsanity.com/paper/2507.10969/) (2025)

## Abstract (arXiv)

Deep convolutional neural networks (CNNs) have shown a strong ability in mining discriminative object pose and parts information for image recognition. For fine-grained recognition, context-aware rich feature representation of object/scene plays a key role since it exhibits a significant variance in the same subcategory and subtle variance among different subcategories. Finding the subtle variance that fully characterizes the object/scene is not straightforward. To address this, we propose a novel context-aware attentional pooling (CAP) that effectively captures subtle changes via sub-pixel gradients, and learns to attend informative integral regions and their importance in discriminating different subcategories without requiring the bounding-box and/or distinguishable part annotations. We also introduce a novel feature encoding by considering the intrinsic consistency between the informativeness of the integral regions and their spatial structures to capture the semantic correlation among them. Our approach is simple yet extremely effective and can be easily applied on top of a standard classification backbone network. We evaluate our approach using six state-of-the-art (SotA) backbone networks and eight benchmark datasets. Our method significantly outperforms the SotA approaches on six datasets and is very competitive with the remaining two.