# The Application of Two-level Attention Models in Deep Convolutional Neural Network for Fine-grained Image Classification

arXiv: https://arxiv.org/abs/1411.6447 · HTML: https://mlsanity.com/paper/1411.6447/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2014.

## Built on by (most cited first, 30 of 67)

- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [Prompt Distribution Learning](https://mlsanity.com/paper/2205.03340/) (2022)
- [Fine-Grained Image Analysis with Deep Learning: A Survey](https://mlsanity.com/paper/2111.06119/) (2021)
- [Transformer-Based Attention Networks for Continuous Pixel-Wise Prediction](https://mlsanity.com/paper/2103.12091/) (2021)
- [Interpretable Image Classification with Differentiable Prototypes Assignment](https://mlsanity.com/paper/2112.02902/) (2021)
- [e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks](https://mlsanity.com/paper/2105.03761/) (2021)
- [Unsupervised Part Discovery from Contrastive Reconstruction](https://mlsanity.com/paper/2111.06349/) (2021)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Feature Fusion Vision Transformer for Fine-Grained Visual Categorization](https://mlsanity.com/paper/2107.02341/) (2021)
- [Learning Super-Features for Image Retrieval](https://mlsanity.com/paper/2201.13182/) (2022)
- [Context-aware Attentional Pooling (CAP) for Fine-grained Visual Classification](https://mlsanity.com/paper/2101.06635/) (2021)
- [Dual Attention-in-Attention Model for Joint Rain Streak and Raindrop Removal](https://mlsanity.com/paper/2103.07051/) (2021)
- [Fine-Grained Object Classification via Self-Supervised Pose Alignment](https://mlsanity.com/paper/2203.15987/) (2022)
- [Multimodality of AI for Education: Towards Artificial General Intelligence](https://mlsanity.com/paper/2312.06037/) (2023)
- [A free lunch from ViT:Adaptive Attention Multi-scale Fusion Transformer for Fine-grained Visual Recognition](https://mlsanity.com/paper/2110.01240/) (2021)
- [Fashionformer: A simple, Effective and Unified Baseline for Human Fashion Segmentation and Recognition](https://mlsanity.com/paper/2204.04654/) (2022)
- [Learning Gabor Texture Features for Fine-Grained Recognition](https://mlsanity.com/paper/2308.05396/) (2023)
- [Continual Attentive Fusion for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/2202.00432/) (2022)
- [Dynamic MLP for Fine-Grained Image Classification by Leveraging Geographical and Temporal Information](https://mlsanity.com/paper/2203.03253/) (2022)
- [Improving Fine-Grained Visual Recognition in Low Data Regimes via Self-Boosting Attention Mechanism](https://mlsanity.com/paper/2208.00617/) (2022)
- [Learning Pairwise Interaction for Generalizable DeepFake Detection](https://mlsanity.com/paper/2302.13288/) (2023)
- [Mask Guided Attention For Fine-Grained Patchy Image Classification](https://mlsanity.com/paper/2102.02771/) (2021)
- [Detection of Deepfake Videos Using Long Distance Attention](https://mlsanity.com/paper/2106.12832/) (2021)
- [Exploring Localization for Self-supervised Fine-grained Contrastive Learning](https://mlsanity.com/paper/2106.15788/) (2021)
- [A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future](https://mlsanity.com/paper/2412.14056/) (2024)
- [Probabilistic Graph Attention Network with Conditional Kernels for Pixel-Wise Prediction](https://mlsanity.com/paper/2101.02843/) (2021)
- [Fine-Grained Few Shot Learning with Foreground Object Transformation](https://mlsanity.com/paper/2109.05719/) (2021)
- [Neuro-Symbolic Learning: Principles and Applications in Ophthalmology](https://mlsanity.com/paper/2208.00374/) (2022)
- [Part-guided Relational Transformers for Fine-grained Visual Recognition](https://mlsanity.com/paper/2212.13685/) (2022)
- [Concept-Centric Transformers: Enhancing Model Interpretability through Object-Centric Concept Learning within a Shared Global Workspace](https://mlsanity.com/paper/2305.15775/) (2023)

## Abstract (arXiv)

Fine-grained classification is challenging because categories can only be discriminated by subtle and local differences. Variances in the pose, scale or rotation usually make the problem more difficult. Most fine-grained classification systems follow the pipeline of finding foreground object or object parts (where) to extract discriminative features (what). In this paper, we propose to apply visual attention to fine-grained classification task using deep neural network. Our pipeline integrates three types of attention: the bottom-up attention that propose candidate patches, the object-level top-down attention that selects relevant patches to a certain object, and the part-level top-down attention that localizes discriminative parts. We combine these attentions to train domain-specific deep nets, then use it to improve both the what and where aspects. Importantly, we avoid using expensive annotations like bounding box or part information from end-to-end. The weak supervision constraint makes our work easier to generalize. We have verified the effectiveness of the method on the subsets of ILSVRC2012 dataset and CUB200_2011 dataset. Our pipeline delivered significant improvements and achieved the best accuracy under the weakest supervision condition. The performance is competitive against other methods that rely on additional annotations.