mlsanityData to 14 Oct 2026

Bayesian-guided Label Mapping for Visual Reprogramming

arXiv 2410.24018 ↗First posted 31 Oct 2024cs.LG cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Visual reprogramming (VR) leverages the intrinsic capabilities of pretrained vision models by adapting their input or output interfaces to solve downstream tasks whose labels (i.e., downstream labels) might be totally different from the labels associated with the pretrained models (i.e., pretrained labels). When adapting the output interface, label mapping methods transform the pretrained labels to downstream labels by establishing a gradient-free one-to-one correspondence between the two sets of labels. However, in this paper, we reveal that one-to-one mappings may overlook the complex relationship between pretrained and downstream labels. Motivated by this observation, we propose a Bayesian-guided Label Mapping (BLM) method. BLM constructs an iteratively-updated probabilistic label mapping matrix, with each element quantifying a pairwise relationship between pretrained and downstream labels. The assignment of values to the constructed matrix is guided by Bayesian conditional probability, considering the joint distribution of the downstream labels and the labels predicted by the pretrained model on downstream samples. Experiments conducted on both pretrained vision models (e.g., ResNeXt) and vision-language models (e.g., CLIP) demonstrate the superior performance of BLM over existing label mapping methods. The success of BLM also offers a probabilistic lens through which to understand and analyze the effectiveness of VR. Our code is available at https://github.com/tmlr-group/BayesianLM.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 52 of its 69 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

52 of 69 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  8. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  9. 2013Describing Textures in the Wildcited by 2,125
  10. 2022Visual Prompt Tuningcited by 1,681
  11. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  12. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
38 more
  1. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  2. 2020Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptationcited by 848
  3. 2022MaPLe: Multi-modal Prompt Learningcited by 687
  4. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 605
  5. 2016Unsupervised Domain Adaptation with Residual Transfer Networkscited by 341
  6. 2022Exploring Visual Prompts for Adapting Large-Scale Modelscited by 274
  7. 2021WARP: Word-level Adversarial ReProgrammingcited by 221
  8. 2022ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learningcited by 193
  9. 2020LEEP: A New Measure to Evaluate Transferability of Learned Representationscited by 188
  10. 2018Adversarial Reprogramming of Neural Networkscited by 116
  11. 2024Low-Rank Few-Shot Adaptation of Vision-Language Modelscited by 104
  12. 2021Voice2Series: Reprogramming Acoustic Models for Time Series Classificationcited by 91
  13. 2022Understanding and Improving Visual Prompting: A Label-Mapping Perspectivecited by 87
  14. 2023BlackVIP: Black-Box Visual Prompting for Robust Transfer Learningcited by 70
  15. 2023E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuningcited by 70
  16. 2023Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inferencecited by 69
  17. 2022Model Reprogramming: Resource-Efficient Cross-Domain Machine Learningcited by 64
  18. 2022Conv-Adapter: Exploring Parameter Efficient Transfer Learning for ConvNetscited by 62
  19. 2020Progressive Graph Learning for Open-Set Domain Adaptationcited by 56
  20. 2022Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learningcited by 53
  21. 2019Open Set Domain Adaptation: Theoretical Bound and Algorithmcited by 48
  22. 2023When Source-Free Domain Adaptation Meets Learning with Noisy Labelscited by 47
  23. 2021Cross-modal Adversarial Reprogrammingcited by 34
  24. 2023MeLo: Low-rank Adaptation is Better than Fine-tuning for Medical Image Diagnosiscited by 33
  25. 2024Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuningcited by 31
  26. 2023AutoVP: An Automated Visual Prompting Framework and Benchmarkcited by 29
  27. 2022Watermarking for Out-of-distribution Detectioncited by 28
  28. 2023Understanding and Mitigating the Label Noise in Pre-training on Downstream Taskscited by 27
  29. 2023From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognitioncited by 23
  30. 2023Deep Graph Reprogrammingcited by 22
  31. 2024Sample-specific Masks for Visual Reprogramming-based Promptingcited by 18
  32. 2021TOHAN: A One-step Approach towards Few-shot Hypothesis Adaptationcited by 17
  33. 2022Low-Resource Music Genre Classification with Cross-Modal Neural Model Reprogrammingcited by 17
  34. 2023TransHP: Image Classification with Hierarchical Promptingcited by 16
  35. 2024Exploring the Transferability of Visual Prompting for Multimodal Large Language Modelscited by 15
  36. 2020Reprogramming Language Models for Molecular Representation Learningcited by 13
  37. 2023Three Towers: Flexible Contrastive Learning with Pretrained Image Modelscited by 11
  38. 2023Cal-SFDA: Source-Free Domain-adaptive Semantic Segmentation with Differentiable Expected Calibration Errorcited by 10

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 4.

12 papers.

  1. 2025Prompt-based Adaptation in Large-scale Vision Models: A Surveycited by 21
  2. 2025Visual Variational Autoencoder Prompt Tuningcited by 10
  3. 2025Attribute-based Visual Reprogramming for Vision-Language Modelscited by 8
  4. 2025Understanding Model Reprogramming for CLIP via Decoupling Visual Promptscited by 4
  5. 2025Neural Network Reprogrammability: A Unified Theme on Model Reprogramming, Prompt Tuning, and Prompt Instructioncited by 4
  6. 2025Vocabulary-free few-shot learning for Vision-Language Modelscited by 1
  7. 2025VirDA: Reusing Backbone for Unsupervised Domain Adaptation with Visual Reprogrammingcited by 1
  8. 2026Let's Roll a BiFTA: Bi-refinement for Fine-grained Text-visual Alignment in Vision-Language Modelscited by 1
  9. 2025Enhancing Visual Prompting through Expanded Transformation Space and Overfitting Mitigationcited by 0
  10. 2026Beyond Low-Rank: Low-Rank Sparse Prompting via Spiking Neural Network and Prompt Factorizationcited by 0
  11. 2026You Only Reprogram Once: Rethinking Prolonged Training for Visual Reprogrammingcited by 0
  12. 2026Reprogramming Vision-Language Models via Structured Prompt Reparameterizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Vision-Language Model Selection and Reuse for Downstream Adaptation
  2. 2024Conditional Prototype Rectification Prompt Learning
  3. 2025M3PO: Multimodal-Model-Guided Preference Optimization for Visual Instruction Following
  4. 2026Multi-Label Test-Time Adaptation with Bayesian Conditional Priors
  5. 2025Visual Instruction Pretraining for Domain-Specific Foundation Models
  6. 2025Seeing to Act, Prompting to Specify: A Bayesian Factorization of Vision Language Action Policy
  7. 2026Pathways of Visual Information Flow in Vision-Language Models
  8. 2022Zero-shot visual reasoning through probabilistic analogical mapping