# Fine-grained Image Retrieval via Dual-Vision Adaptation

arXiv: https://arxiv.org/abs/2506.16273 · HTML: https://mlsanity.com/paper/2506.16273/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 52 references that are in the map.
- First posted on arXiv 19 Jun 2025.

## Builds on (most cited first, 30 of 31)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition](https://mlsanity.com/paper/2205.13535/) (2022)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks](https://mlsanity.com/paper/2004.05937/) (2020)
- [No Fuss Distance Metric Learning using Proxies](https://mlsanity.com/paper/1703.07464/) (2017)
- [Proxy Anchor Loss for Deep Metric Learning](https://mlsanity.com/paper/2003.13911/) (2020)
- [Benchmarking Representation Learning for Natural World Image Collections](https://mlsanity.com/paper/2103.16483/) (2021)
- [Hyperbolic Vision Transformers: Combining Improvements in Metric Learning](https://mlsanity.com/paper/2203.10833/) (2022)
- [ProxyNCA++: Revisiting and Revitalizing Proxy Neighborhood Component Analysis](https://mlsanity.com/paper/2004.01113/) (2020)
- [StyleShot: A Snapshot on Any Style](https://mlsanity.com/paper/2407.01414/) (2024)
- [DarkRank: Accelerating Deep Metric Learning via Cross Sample Similarities Transfer](https://mlsanity.com/paper/1707.01220/) (2017)
- [Bird Species Categorization Using Pose Normalized Deep Convolutional Nets](https://mlsanity.com/paper/1406.2952/) (2014)
- [Selective Convolutional Descriptor Aggregation for Fine-Grained Image Retrieval](https://mlsanity.com/paper/1604.04994/) (2016)
- [Learning Intra-Batch Connections for Deep Metric Learning](https://mlsanity.com/paper/2102.07753/) (2021)
- [M3Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action Recognition](https://mlsanity.com/paper/2308.03063/) (2023)
- [Teacher-Student Architecture for Knowledge Distillation: A Survey](https://mlsanity.com/paper/2308.04268/) (2023)
- [Attribute-Aware Attention Model for Fine-grained Representation Learning](https://mlsanity.com/paper/1901.00392/) (2019)
- [Keypoint-Aligned Embeddings for Image Retrieval and Re-identification](https://mlsanity.com/paper/2008.11368/) (2020)
- [FaceShot: Bring Any Character into Life](https://mlsanity.com/paper/2503.00740/) (2025)
- [FodFoM: Fake Outlier Data by Foundation Models Creates Stronger Visual Out-of-Distribution Detector](https://mlsanity.com/paper/2412.05293/) (2024)
- [Learning Contrastive Self-Distillation for Ultra-Fine-Grained Visual Categorization Targeting Limited Samples](https://mlsanity.com/paper/2311.06056/) (2023)
- [DVF: Advancing Robust and Accurate Fine-Grained Image Retrieval with Retrieval Guidelines](https://mlsanity.com/paper/2404.15771/) (2024)
- [AdaFPP: Adapt-Focused Bi-Propagating Prototype Learning for Panoramic Activity Recognition](https://mlsanity.com/paper/2405.02538/) (2024)
- [EventCrab: Harnessing Frame and Point Synergy for Event-based Action Recognition and Beyond](https://mlsanity.com/paper/2411.18328/) (2024)

## Built on by (most cited first, 2 of 2)

- [Spatiotemporal-Untrammelled Mixture of Experts for Multi-Person Motion Prediction](https://mlsanity.com/paper/2512.21707/) (2025)
- [Learning to Align Generative Appearance Priors for Fine-grained Image Retrieval](https://mlsanity.com/paper/2605.09859/) (2026)

## Abstract (arXiv)

Fine-Grained Image Retrieval~(FGIR) faces challenges in learning discriminative visual representations to retrieve images with similar fine-grained features. Current leading FGIR solutions typically follow two regimes: enforce pairwise similarity constraints in the semantic embedding space, or incorporate a localization sub-network to fine-tune the entire model. However, such two regimes tend to overfit the training data while forgetting the knowledge gained from large-scale pre-training, thus reducing their generalization ability. In this paper, we propose a Dual-Vision Adaptation (DVA) approach for FGIR, which guides the frozen pre-trained model to perform FGIR through collaborative sample and feature adaptation. Specifically, we design Object-Perceptual Adaptation, which modifies input samples to help the pre-trained model perceive critical objects and elements within objects that are helpful for category prediction. Meanwhile, we propose In-Context Adaptation, which introduces a small set of parameters for feature adaptation without modifying the pre-trained parameters. This makes the FGIR task using these adjusted features closer to the task solved during the pre-training. Additionally, to balance retrieval efficiency and performance, we propose Discrimination Perception Transfer to transfer the discriminative knowledge in the object-perceptual adaptation to the image encoder using the knowledge distillation mechanism. Extensive experiments show that DVA has fewer learnable parameters and performs well on three in-distribution and three out-of-distribution fine-grained datasets.