# RAMS-Trans: Recurrent Attention Multi-scale Transformer forFine-grained Image Recognition

arXiv: https://arxiv.org/abs/2107.08192 · HTML: https://mlsanity.com/paper/2107.08192/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 50 references that are in the map.
- First posted on arXiv 17 Jul 2021.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Bilinear CNNs for Fine-grained Visual Recognition](https://mlsanity.com/paper/1504.07889/) (2015)
- [TransFG: A Transformer Architecture for Fine-grained Recognition](https://mlsanity.com/paper/2103.07976/) (2021)
- [Learning to Navigate for Fine-grained Classification](https://mlsanity.com/paper/1809.00287/) (2018)
- [Compact Bilinear Pooling](https://mlsanity.com/paper/1511.06062/) (2015)
- [Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognition](https://mlsanity.com/paper/1903.06150/) (2019)
- [Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patches](https://mlsanity.com/paper/2003.03836/) (2020)
- [Learning Attentive Pairwise Interaction for Fine-Grained Classification](https://mlsanity.com/paper/2002.10191/) (2020)
- [Learn To Pay Attention](https://mlsanity.com/paper/1804.02391/) (2018)
- [Interpretable and Accurate Fine-grained Recognition via Region Grouping](https://mlsanity.com/paper/2005.10411/) (2020)
- [Weakly Supervised Complementary Parts Models for Fine-Grained Image Classification from the Bottom Up](https://mlsanity.com/paper/1903.02827/) (2019)
- [Maximum-Entropy Fine-Grained Classification](https://mlsanity.com/paper/1809.05934/) (2018)
- [Cross-X Learning for Fine-Grained Visual Categorization](https://mlsanity.com/paper/1909.04412/) (2019)
- [Learning to Zoom: a Saliency-Based Sampling Layer for Neural Networks](https://mlsanity.com/paper/1809.03355/) (2018)
- [Learning Deep Bilinear Transformation for Fine-grained Image Representation](https://mlsanity.com/paper/1911.03621/) (2019)
- [Attention Convolutional Binary Neural Tree for Fine-Grained Visual Categorization](https://mlsanity.com/paper/1909.11378/) (2019)
- [Bird Species Categorization Using Pose Normalized Deep Convolutional Nets](https://mlsanity.com/paper/1406.2952/) (2014)

## Built on by (most cited first, 30 of 34)

- [Dual Cross-Attention Learning for Fine-Grained Visual Categorization and Object Re-Identification](https://mlsanity.com/paper/2205.02151/) (2022)
- [Which Tokens to Use? Investigating Token Reduction in Vision Transformers](https://mlsanity.com/paper/2308.04657/) (2023)
- [Fine-grained Visual Classification with High-temperature Refinement and Background Suppression](https://mlsanity.com/paper/2303.06442/) (2023)
- [SIM-Trans: Structure Information Modeling Transformer for Fine-grained Visual Categorization](https://mlsanity.com/paper/2208.14607/) (2022)
- [A free lunch from ViT:Adaptive Attention Multi-scale Fusion Transformer for Fine-grained Visual Recognition](https://mlsanity.com/paper/2110.01240/) (2021)
- [Delving into Multimodal Prompting for Fine-grained Visual Classification](https://mlsanity.com/paper/2309.08912/) (2023)
- [Agglomerative Token Clustering](https://mlsanity.com/paper/2409.11923/) (2024)
- [A Novel Plug-in Module for Fine-Grained Visual Classification](https://mlsanity.com/paper/2202.03822/) (2022)
- [Learning Contrastive Self-Distillation for Ultra-Fine-Grained Visual Categorization Targeting Limited Samples](https://mlsanity.com/paper/2311.06056/) (2023)
- [R2-Trans:Fine-Grained Visual Categorization with Redundancy Reduction](https://mlsanity.com/paper/2204.10095/) (2022)
- [Learning Fine-grained Domain Generalization via Hyperbolic State Space Hallucination](https://mlsanity.com/paper/2504.08020/) (2025)
- [Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition](https://mlsanity.com/paper/2305.07180/) (2023)
- [Coping with Change: Learning Invariant and Minimum Sufficient Representations for Fine-Grained Visual Categorization](https://mlsanity.com/paper/2306.04893/) (2023)
- [Global-Local Similarity for Efficient Fine-Grained Image Recognition with Vision Transformers](https://mlsanity.com/paper/2407.12891/) (2024)
- [MetaVideoAgent: Automated Video-Agent Evolution for Long-Form Video Understanding](https://mlsanity.com/paper/2608.04587/) (2026)
- [M2Former: Multi-Scale Patch Selection for Fine-Grained Visual Recognition](https://mlsanity.com/paper/2308.02161/) (2023)
- [Learning Time in Static Classifiers](https://mlsanity.com/paper/2511.12321/) (2025)
- [ViT-FOD: A Vision Transformer based Fine-grained Object Discriminator](https://mlsanity.com/paper/2203.12816/) (2022)
- [Conviformers: Convolutionally guided Vision Transformer](https://mlsanity.com/paper/2208.08900/) (2022)
- [ExpNet: A unified network for Expert-Level Classification](https://mlsanity.com/paper/2211.15672/) (2022)
- [Multi-scale multi-modal micro-expression recognition algorithm based on transformer](https://mlsanity.com/paper/2301.02969/) (2023)
- [Cross-Level Multi-Instance Distillation for Self-Supervised Fine-Grained Visual Categorization](https://mlsanity.com/paper/2401.08860/) (2024)
- [Down-Sampling Inter-Layer Adapter for Parameter and Computation Efficient Ultra-Fine-Grained Image Recognition](https://mlsanity.com/paper/2409.11051/) (2024)
- [MCFNet: A Multimodal Collaborative Fusion Network for Fine-Grained Semantic Classification](https://mlsanity.com/paper/2505.23365/) (2025)
- [PicT: A Slim Weakly Supervised Vision Transformer for Pavement Distress Classification](https://mlsanity.com/paper/2209.10074/) (2022)
- [Semantic Feature Integration network for Fine-grained Visual Classification](https://mlsanity.com/paper/2302.10275/) (2023)
- [Improving Deep Representation Learning via Auxiliary Learnable Target Coding](https://mlsanity.com/paper/2305.18680/) (2023)
- [Recent Advances of Local Mechanisms in Computer Vision: A Survey and Outlook of Recent Work](https://mlsanity.com/paper/2306.01929/) (2023)
- [Not just Birds and Cars: Generic, Scalable and Explainable Models for Professional Visual Recognition](https://mlsanity.com/paper/2403.05703/) (2024)
- [DCNN: Dual Cross-current Neural Networks Realized Using An Interactive Deep Learning Discriminator for Fine-grained Objects](https://mlsanity.com/paper/2405.04093/) (2024)

## Abstract (arXiv)

In fine-grained image recognition (FGIR), the localization and amplification of region attention is an important factor, which has been explored a lot by convolutional neural networks (CNNs) based approaches. The recently developed vision transformer (ViT) has achieved promising results on computer vision tasks. Compared with CNNs, Image sequentialization is a brand new manner. However, ViT is limited in its receptive field size and thus lacks local attention like CNNs due to the fixed size of its patches, and is unable to generate multi-scale features to learn discriminative region attention. To facilitate the learning of discriminative region attention without box/part annotations, we use the strength of the attention weights to measure the importance of the patch tokens corresponding to the raw images. We propose the recurrent attention multi-scale transformer (RAMS-Trans), which uses the transformer's self-attention to recursively learn discriminative region attention in a multi-scale manner. Specifically, at the core of our approach lies the dynamic patch proposal module (DPPM) guided region amplification to complete the integration of multi-scale image patches. The DPPM starts with the full-size image patches and iteratively scales up the region attention to generate new patches from global to local by the intensity of the attention weights generated at each scale as an indicator. Our approach requires only the attention weights that come with ViT itself and can be easily trained end-to-end. Extensive experiments demonstrate that RAMS-Trans performs better than concurrent works, in addition to efficient CNN models, achieving state-of-the-art results on three benchmark datasets.