# MetaFormer: A Unified Meta Framework for Fine-Grained Recognition

arXiv: https://arxiv.org/abs/2203.02751 · HTML: https://mlsanity.com/paper/2203.02751/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 58 references that are in the map.
- First posted on arXiv 5 Mar 2022.

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [PVT v2: Improved Baselines with Pyramid Vision Transformer](https://mlsanity.com/paper/2106.13797/) (2021)
- [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021)
- [CoAtNet: Marrying Convolution and Attention for All Data Sizes](https://mlsanity.com/paper/2106.04803/) (2021)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [Sparse R-CNN: End-to-End Object Detection with Learnable Proposals](https://mlsanity.com/paper/2011.12450/) (2020)
- [ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases](https://mlsanity.com/paper/2103.10697/) (2021)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [TransTrack: Multiple Object Tracking with Transformer](https://mlsanity.com/paper/2012.15460/) (2020)
- [Bilinear CNNs for Fine-grained Visual Recognition](https://mlsanity.com/paper/1504.07889/) (2015)
- [Fixing the train-test resolution discrepancy](https://mlsanity.com/paper/1906.06423/) (2019)
- [UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training](https://mlsanity.com/paper/2002.12804/) (2020)
- [Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning](https://mlsanity.com/paper/1806.06193/) (2018)
- [TransFG: A Transformer Architecture for Fine-grained Recognition](https://mlsanity.com/paper/2103.07976/) (2021)
- [Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identification](https://mlsanity.com/paper/2108.08728/) (2021)
- [Presence-Only Geographical Priors for Fine-Grained Image Classification](https://mlsanity.com/paper/1906.05272/) (2019)
- [Learning to Navigate for Fine-grained Classification](https://mlsanity.com/paper/1809.00287/) (2018)
- [Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognition](https://mlsanity.com/paper/1903.06150/) (2019)

## Built on by (most cited first, 28 of 28)

- [InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions](https://mlsanity.com/paper/2211.05778/) (2022)
- [MTS-Mixers: Multivariate Time Series Forecasting via Factorized Temporal and Channel Mixing](https://mlsanity.com/paper/2302.04501/) (2023)
- [TaxaBind: A Unified Embedding Space for Ecological Applications](https://mlsanity.com/paper/2411.00683/) (2024)
- [Federated Generative Learning with Foundation Models](https://mlsanity.com/paper/2306.16064/) (2023)
- [Fine-grained Visual Classification with High-temperature Refinement and Background Suppression](https://mlsanity.com/paper/2303.06442/) (2023)
- [Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation](https://mlsanity.com/paper/2504.14988/) (2025)
- [BirdSAT: Cross-View Contrastive Masked Autoencoders for Bird Species Classification and Mapping](https://mlsanity.com/paper/2310.19168/) (2023)
- [FungiTastic: A multi-modal dataset and benchmark for image categorization](https://mlsanity.com/paper/2408.13632/) (2024)
- [U-Netmer: U-Net meets Transformer for medical image segmentation](https://mlsanity.com/paper/2304.01401/) (2023)
- [Salient Mask-Guided Vision Transformer for Fine-Grained Classification](https://mlsanity.com/paper/2305.07102/) (2023)
- [Fine-Grained Classification for Poisonous Fungi Identification with Transfer Learning](https://mlsanity.com/paper/2407.07492/) (2024)
- [Solutions for Fine-grained and Long-tailed Snake Species Recognition in SnakeCLEF 2022](https://mlsanity.com/paper/2207.01216/) (2022)
- [A Dual-scale Lead-seperated Transformer With Lead-orthogonal Attention And Meta-information For Ecg Classification](https://mlsanity.com/paper/2211.12777/) (2022)
- [UVL2: A Unified Framework for Video Tampering Localization](https://mlsanity.com/paper/2309.16126/) (2023)
- [Learning from the few: Fine-grained approach to pediatric wrist pathology recognition on a limited dataset](https://mlsanity.com/paper/2408.13542/) (2024)
- [Cross-Hierarchical Bidirectional Consistency Learning for Fine-Grained Visual Classification](https://mlsanity.com/paper/2504.13608/) (2025)
- [Explored An Effective Methodology for Fine-Grained Snake Recognition](https://mlsanity.com/paper/2207.11637/) (2022)
- [Maintaining Performance with Less Data](https://mlsanity.com/paper/2208.02007/) (2022)
- [Leveraging Habitat Information for Fine-grained Bird Identification](https://mlsanity.com/paper/2312.14999/) (2023)
- [Probabilistic Image-Driven Traffic Modeling via Remote Sensing](https://mlsanity.com/paper/2403.05521/) (2024)
- [MARs: Multi-view Attention Regularizations for Patch-based Feature Recognition of Space Terrain](https://mlsanity.com/paper/2410.05182/) (2024)
- [SGIA: Enhancing Fine-Grained Visual Classification with Sequence Generative Image Augmentation](https://mlsanity.com/paper/2412.06138/) (2024)
- [Navigating limitations with precision: A fine-grained ensemble approach to wrist pathology recognition on a limited x-ray dataset](https://mlsanity.com/paper/2412.13884/) (2024)
- [CrypticBio: A Large Multimodal Dataset for Visually Confusing Biodiversity](https://mlsanity.com/paper/2505.14707/) (2025)
- [AquaMonitor: A multimodal multi-view image sequence dataset for real-life aquatic invertebrate biodiversity monitoring](https://mlsanity.com/paper/2505.22065/) (2025)
- [Towards Continuous Home Cage Monitoring: An Evaluation of Tracking and Identification Strategies for Laboratory Mice](https://mlsanity.com/paper/2507.07929/) (2025)
- [Demographic-aware fine-grained visual recognition of pediatric wrist pathologies](https://mlsanity.com/paper/2507.12964/) (2025)
- [Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: From Evaluation to Diagnosis](https://mlsanity.com/paper/2606.19053/) (2026)

## Abstract (arXiv)

Fine-Grained Visual Classification(FGVC) is the task that requires recognizing the objects belonging to multiple subordinate categories of a super-category. Recent state-of-the-art methods usually design sophisticated learning pipelines to tackle this task. However, visual information alone is often not sufficient to accurately differentiate between fine-grained visual categories. Nowadays, the meta-information (e.g., spatio-temporal prior, attribute, and text description) usually appears along with the images. This inspires us to ask the question: Is it possible to use a unified and simple framework to utilize various meta-information to assist in fine-grained identification? To answer this problem, we explore a unified and strong meta-framework(MetaFormer) for fine-grained visual classification. In practice, MetaFormer provides a simple yet effective approach to address the joint learning of vision and various meta-information. Moreover, MetaFormer also provides a strong baseline for FGVC without bells and whistles. Extensive experiments demonstrate that MetaFormer can effectively use various meta-information to improve the performance of fine-grained recognition. In a fair comparison, MetaFormer can outperform the current SotA approaches with only vision information on the iNaturalist2017 and iNaturalist2018 datasets. Adding meta-information, MetaFormer can exceed the current SotA approaches by 5.9% and 5.3%, respectively. Moreover, MetaFormer can achieve 92.3% and 92.7% on CUB-200-2011 and NABirds, which significantly outperforms the SotA approaches. The source code and pre-trained models are released athttps://github.com/dqshuai/MetaFormer.