Data to 7 Oct 2026

Retrieval Augmented Classification for Long-Tail Visual Recognition

arXiv 2202.11233 ↗First posted 22 Feb 2022cs.CVcited by 106

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

We introduce Retrieval Augmented Classification (RAC), a generic approach to augmenting standard image classification pipelines with an explicit retrieval module. RAC consists of a standard base image encoder fused with a parallel retrieval branch that queries a non-parametric external memory of pre-encoded images and associated text snippets. We apply RAC to the problem of long-tail classification and demonstrate a significant improvement over previous state-of-the-art on Places365-LT and iNaturalist-2018 (14.5% and 6.7% respectively), despite using only the training datasets themselves as the external information source. We demonstrate that RAC's retrieval module, without prompting, learns a high level of accuracy on tail classes. This, in turn, frees the base encoder to focus on common classes, and improve its performance thereon. RAC represents an alternative approach to utilizing large, pretrained models without requiring fine-tuning, as well as a first step towards more effectively making use of external memory within common computer vision architectures.

Key facts. Cited by 106 papers in the map (data through 7 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 44 of its 66 references that are in the map. First posted on arXiv 22 Feb 2022.

Builds on

44 of 66 references.

References · 44

2017Attention Is All You Needcited by 59,932
2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
2018Representation Learning with Contrastive Predictive Codingcited by 7,662
2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
2017On Calibration of Modern Neural Networkscited by 4,860
2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
2017Billion-scale similarity search with GPUscited by 2,477
2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239

32 more

2021Transformers in Vision: A Surveycited by 1,009
2021Scaling Vision Transformerscited by 993
2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
2019When Does Label Smoothing Help?cited by 985
2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
2019Big Transfer (BiT): General Visual Representation Learningcited by 766
2017Focal Loss for Dense Object Detectioncited by 749
2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
2014Neural Turing Machinescited by 630
2020Transformer Interpretability Beyond Attention Visualizationcited by 620
2020Long-tail learning via logit adjustmentcited by 605
2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
2019Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detectioncited by 560
2021Editing Factual Knowledge in Language Modelscited by 548
2019BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognitioncited by 403
2021ResNet strikes back: An improved training procedure in timmcited by 365
2020Long-tailed Recognition by Routing Diverse Distribution-Aware Expertscited by 270
2021Parametric Contrastive Learningcited by 217
2021Distribution Alignment: A Unified Framework for Long-tail Visual Recognitioncited by 198
2020Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classificationcited by 175
2020Disentangling Label Distribution for Long-tailed Visual Recognitioncited by 172
2020Feature Space Augmentation for Long-Tailed Datacited by 136
2020Deep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspectivecited by 120
2016Range Loss for Deep Face Recognition with Long-tailcited by 111
2020Grafit: Learning fine-grained image representations with coarse labelscited by 43
2021Adaptive Logit Adjustment Loss for Long-Tailed Visual Recognitioncited by 33
2021PML: Progressive Margin Loss for Long-tailed Age Classificationcited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 12; 2023: 27; 2024: 35; 2025: 21; 2026: 11.

106 papers.

Sort:

By area:

2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 208
2022REVEAL: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memorycited by 122
2023Lift Yourself Up: Retrieval-augmented Text Generation with Self Memorycited by 103
2023TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023cited by 73
2022LPT: Long-tailed Prompt Tuning for Image Classificationcited by 69
2022Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Modelscited by 63
2023Learning Customized Visual Models with Retrieval-Augmented Knowledgecited by 58
2023Long-Tail Learning with Foundation Model: Heavy Fine-Tuning Hurtscited by 58
2024Retrieval-Augmented Egocentric Video Captioningcited by 42
2023Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioningcited by 41
2022Semi-Parametric Neural Image Synthesiscited by 36
2023Retrieval-Enhanced Contrastive Vision-Text Modelscited by 36

38 more

2024Knowledge-Enhanced Dual-stream Zero-shot Composed Image Retrievalcited by 36
2023Vocabulary-free Image Classificationcited by 35
2024MeaCap: Memory-Augmented Zero-shot Image Captioningcited by 35
2022Learning Imbalanced Data with Vision Transformerscited by 34
2024LTGC: Long-tail Recognition via Leveraging LLMs-driven Generated Contentcited by 33
2023Towards Generalisable Video Moment Retrieval: Visual-Dynamic Injection to Image-Text Pre-Trainingcited by 32
2025Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlookcited by 32
2023RAMM: Retrieval-augmented Biomedical Visual Question Answering with Multi-modal Pre-trainingcited by 31
2024OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretrainingcited by 29
2023Retrieving-to-Answer: Zero-Shot Video Question Answering with Frozen Large Language Modelscited by 27
2024A Systematic Review on Long-Tailed Learningcited by 27
2022Revisiting a kNN-based Image Classification System with High-capacity Storagecited by 22
2022Memories are One-to-Many Mapping Alleviators in Talking Face Generationcited by 22
2023Unbiased Scene Graph Generation via Two-stage Causal Modelingcited by 21
2023Improving Image Recognition by Retrieving from Web-Scale Image-Text Datacited by 20
2023Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labelscited by 20
2024Retrieval-Augmented Open-Vocabulary Object Detectioncited by 20
2024Retrieval-augmented Few-shot Medical Image Segmentation with Foundation Modelscited by 20
2023MDCS: More Diverse Experts with Consistency Self-distillation for Long-tailed Recognitioncited by 19
2025RANGE: Retrieval Augmented Neural Fields for Multi-Resolution Geo-Embeddingscited by 18
2024Data-Centric Evolution in Autonomous Driving: A Comprehensive Survey of Big Data System, Data Mining, and Closed-Loop Technologiescited by 17
2024Improving Visual Prompt Tuning by Gaussian Neighborhood Minimization for Long-Tailed Visual Recognitioncited by 15
2024DeiT-LT Distillation Strikes Back for Vision Transformer Training on Long-Tailed Datasetscited by 14
2025A Causal Adjustment Module for Debiasing Scene Graph Generationcited by 14
2024RoarGraph: A Projected Bipartite Graph for Efficient Cross-Modal Approximate Nearest Neighbor Searchcited by 12
2024Uncertainty-Guided Appearance-Motion Association Network for Out-of-Distribution Action Detectioncited by 12
2024Retrieval-Enhanced Machine Learning: Synthesis and Opportunitiescited by 11
2024BEV-TSR: Text-Scene Retrieval in BEV Space for Autonomous Drivingcited by 10
2024Understanding Retrieval-Augmented Task Adaptation for Vision-Language Modelscited by 10
2023NICE: CVPR 2023 Challenge on Zero-shot Image Captioningcited by 9
2024Diffusion Models For Multi-Modal Generative Modelingcited by 9
2024Retrieval Augmented Diffusion Model for Structure-informed Antibody Design and Optimizationcited by 9
2024Learning from Neighbors: Category Extrapolation for Long-Tail Learningcited by 9
2022The Equalization Losses: Gradient-Driven Training for Long-tailed Object Recognitioncited by 8
2023Continual Learning in Open-vocabulary Classification with Complementary Memory Systemscited by 8
2024Vocabulary-free Image Classification and Semantic Segmentationcited by 8
2023Train/Test-Time Adaptation with Retrievalcited by 7
2023Exploring Weight Balancing on Long-Tailed Recognition Problemcited by 7

50 most cited of 106 All 106 (JSON)

Similar, not cited

Close in wording, no citation either way