# Improving Fine-Grained Visual Recognition in Low Data Regimes via Self-Boosting Attention Mechanism

arXiv: https://arxiv.org/abs/2208.00617 · HTML: https://mlsanity.com/paper/2208.00617/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 55 references that are in the map.
- First posted on arXiv 1 Aug 2022.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Bilinear CNNs for Fine-grained Visual Recognition](https://mlsanity.com/paper/1504.07889/) (2015)
- [Weakly Supervised Deep Detection Networks](https://mlsanity.com/paper/1511.02853/) (2015)
- [Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning](https://mlsanity.com/paper/1806.06193/) (2018)
- [Fine-Grained Image Analysis with Deep Learning: A Survey](https://mlsanity.com/paper/2111.06119/) (2021)
- [A Large-Scale Car Dataset for Fine-Grained Categorization and Verification](https://mlsanity.com/paper/1506.08959/) (2015)
- [Return of the Devil in the Details: Delving Deep into Convolutional Nets](https://mlsanity.com/paper/1405.3531/) (2014)
- [Part-based R-CNNs for Fine-grained Category Detection](https://mlsanity.com/paper/1407.3867/) (2014)
- [Compact Bilinear Pooling](https://mlsanity.com/paper/1511.06062/) (2015)
- [Learning Attentive Pairwise Interaction for Fine-Grained Classification](https://mlsanity.com/paper/2002.10191/) (2020)
- [The Application of Two-level Attention Models in Deep Convolutional Neural Network for Fine-grained Image Classification](https://mlsanity.com/paper/1411.6447/) (2014)
- [Part-Stacked CNN for Fine-Grained Visual Categorization](https://mlsanity.com/paper/1512.08086/) (2015)
- [See Better Before Looking Closer: Weakly Supervised Data Augmentation Network for Fine-Grained Visual Classification](https://mlsanity.com/paper/1901.09891/) (2019)
- [Learning Deep Bilinear Transformation for Fine-grained Image Representation](https://mlsanity.com/paper/1911.03621/) (2019)
- [Bird Species Categorization Using Pose Normalized Deep Convolutional Nets](https://mlsanity.com/paper/1406.2952/) (2014)
- [The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition](https://mlsanity.com/paper/1511.06789/) (2015)
- [Hierarchical Bilinear Pooling for Fine-Grained Visual Recognition](https://mlsanity.com/paper/1807.09915/) (2018)
- [Low-rank Bilinear Pooling for Fine-Grained Classification](https://mlsanity.com/paper/1611.05109/) (2016)
- [Self-Tuning for Data-Efficient Deep Learning](https://mlsanity.com/paper/2102.12903/) (2021)
- [Piecewise classifier mappings: Learning fine-grained learners for novel categories with few examples](https://mlsanity.com/paper/1805.04288/) (2018)
- [Attention for Fine-Grained Categorization](https://mlsanity.com/paper/1412.7054/) (2014)
- [Multi-Objective Matrix Normalization for Fine-grained Visual Recognition](https://mlsanity.com/paper/2003.13272/) (2020)

## Built on by (most cited first, 12 of 12)

- [Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems](https://mlsanity.com/paper/2303.01669/) (2023)
- [On Learning Discriminative Features from Synthesized Data for Self-Supervised Fine-Grained Visual Recognition](https://mlsanity.com/paper/2407.14676/) (2024)
- [Detail Reinforcement Diffusion Model: Augmentation Fine-Grained Visual Categorization in Few-Shot Conditions](https://mlsanity.com/paper/2309.08097/) (2023)
- [Distilling Local Texture Features for Colorectal Tissue Classification in Low Data Regimes](https://mlsanity.com/paper/2401.01164/) (2024)
- [Cross-Level Multi-Instance Distillation for Self-Supervised Fine-Grained Visual Categorization](https://mlsanity.com/paper/2401.08860/) (2024)
- [Source-Free Unsupervised Domain Adaptation with Hypothesis Consolidation of Prediction Rationale](https://mlsanity.com/paper/2402.01157/) (2024)
- [LoDisc: Learning Global-Local Discriminative Features for Self-Supervised Fine-Grained Visual Recognition](https://mlsanity.com/paper/2403.04066/) (2024)
- [Extract More from Less: Efficient Fine-Grained Visual Recognition in Low-Data Regimes](https://mlsanity.com/paper/2406.19814/) (2024)
- [PP-SSL : Priority-Perception Self-Supervised Learning for Fine-Grained Recognition](https://mlsanity.com/paper/2412.00134/) (2024)
- [Progressive Feature Adjustment for Semi-supervised Learning from Pretrained Models](https://mlsanity.com/paper/2309.04659/) (2023)
- [Enhancing Fine-Grained Visual Recognition in the Low-Data Regime Through Feature Magnitude Regularization](https://mlsanity.com/paper/2409.01672/) (2024)
- [LLMs-based Augmentation for Domain Adaptation in Long-tailed Food Datasets](https://mlsanity.com/paper/2511.16037/) (2025)

## Abstract (arXiv)

The challenge of fine-grained visual recognition often lies in discovering the key discriminative regions. While such regions can be automatically identified from a large-scale labeled dataset, a similar method might become less effective when only a few annotations are available. In low data regimes, a network often struggles to choose the correct regions for recognition and tends to overfit spurious correlated patterns from the training data. To tackle this issue, this paper proposes the self-boosting attention mechanism, a novel method for regularizing the network to focus on the key regions shared across samples and classes. Specifically, the proposed method first generates an attention map for each training image, highlighting the discriminative part for identifying the ground-truth object category. Then the generated attention maps are used as pseudo-annotations. The network is enforced to fit them as an auxiliary task. We call this approach the self-boosting attention mechanism (SAM). We also develop a variant by using SAM to create multiple attention maps to pool convolutional maps in a style of bilinear pooling, dubbed SAM-Bilinear. Through extensive experimental studies, we show that both methods can significantly improve fine-grained visual recognition performance on low data regimes and can be incorporated into existing network architectures. The source code is publicly available at: https://github.com/GANPerf/SAM