M2Former: Multi-Scale Patch Selection for Fine-Grained Visual Recognition
Not on the map. Why?
Abstract (arXiv)
Recently, vision Transformers (ViTs) have been actively applied to fine-grained visual recognition (FGVR). ViT can effectively model the interdependencies between patch-divided object regions through an inherent self-attention mechanism. In addition, patch selection is used with ViT to remove redundant patch information and highlight the most discriminative object patches. However, existing ViT-based FGVR models are limited to single-scale processing, and their fixed receptive fields hinder representational richness and exacerbate vulnerability to scale variability. Therefore, we propose multi-scale patch selection (MSPS) to improve the multi-scale capabilities of existing ViT-based models. Specifically, MSPS selects salient patches of different scales at different stages of a multi-scale vision Transformer (MS-ViT). In addition, we introduce class token transfer (CTT) and multi-scale cross-attention (MSCA) to model cross-scale interactions between selected multi-scale patches and fully reflect them in model decisions. Compared to previous single-scale patch selection (SSPS), our proposed MSPS encourages richer object representations based on feature hierarchy and consistently improves performance from small-sized to large-sized objects. As a result, we propose M2Former, which outperforms CNN-/ViT-based models on several widely used FGVR benchmarks.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 66 of its 95 references that are in the map. First posted on arXiv 4 Aug 2023.
Builds on
66 of 95 references.
References · 66
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2016Feature Pyramid Networks for Object Detectioncited by 5,021
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
- 2017Squeeze-and-Excitation Networkscited by 4,763
38 more
- 2018YOLOv3: An Incremental Improvementcited by 2,652
- 2018CBAM: Convolutional Block Attention Modulecited by 2,463
- 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
- 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
- 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
- 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
- 2019When Does Label Smoothing Help?cited by 985
- 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
- 2021Multiscale Vision Transformerscited by 896
- 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 719
- 2015Holistically-Nested Edge Detectioncited by 590
- 2019ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networkscited by 572
- 2014Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFscited by 552
- 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
- 2022MaxViT: Multi-Axis Vision Transformercited by 476
- 2019Res2Net: A New Multi-scale Backbone Architecturecited by 468
- 2019Is Attention Interpretable?cited by 387
- 2017Dilated Residual Networkscited by 293
- 2018Context Encoding for Semantic Segmentationcited by 289
- 2022Neighborhood Attention Transformercited by 235
- 2019Fixing the train-test resolution discrepancycited by 209
- 2021UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-wise Perspective with Transformercited by 197
- 2023BiFormer: Vision Transformer with Bi-Level Routing Attentioncited by 150
- 2016Richer Convolutional Features for Edge Detectioncited by 141
- 2018Gather-Excite: Exploiting Feature Context in Convolutional Neural Networkscited by 138
- 2021TransFG: A Transformer Architecture for Fine-grained Recognitioncited by 131
- 2017Single-Shot Refinement Neural Network for Object Detectioncited by 129
- 2014Part-based R-CNNs for Fine-grained Category Detectioncited by 107
- 2018Learning to Navigate for Fine-grained Classificationcited by 103
- 2019Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognitioncited by 95
- 2020Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patchescited by 92
- 2021TransFER: Learning Relation-aware Facial Expression Representations with Transformerscited by 85
- 2020Learning Attentive Pairwise Interaction for Fine-Grained Classificationcited by 84
- 2022Dual Cross-Attention Learning for Fine-Grained Visual Categorization and Object Re-Identificationcited by 78
- 2018M2Det: A Single-Shot Object Detector based on Multi-Level Feature Pyramid Networkcited by 76
- 2020Interpretable and Accurate Fine-grained Recognition via Region Groupingcited by 75
- 2014The Application of Two-level Attention Models in Deep Convolutional Neural Network for Fine-grained Image Classificationcited by 67
- 2019Weakly Supervised Complementary Parts Models for Fine-Grained Image Classification from the Bottom Upcited by 66
50 most cited of 66 All 66 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.