mlsanityData to 7 Oct 2026

Zero-Shot Continuous Prompt Transfer: Generalizing Task Semantics Across Language Models

arXiv 2310.01691 ↗First posted 2 Oct 2023cs.CL cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Prompt tuning in natural language processing (NLP) has become an increasingly popular method for adapting large language models to specific tasks. However, the transferability of these prompts, especially continuous prompts, between different models remains a challenge. In this work, we propose a zero-shot continuous prompt transfer method, where source prompts are encoded into relative space and the corresponding target prompts are searched for transferring to target models. Experimental results confirm the effectiveness of our method, showing that 'task semantics' in continuous prompts can be generalized across various language models. Moreover, we find that combining 'task semantics' from multiple source models can further enhance the generalizability of transfer.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 29 of its 35 references that are in the map. First posted on arXiv 2 Oct 2023.

Builds on

29 of 35 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  9. 2019Language Models as Knowledge Bases?cited by 1,922
  10. 2021Learning to Prompt for Continual Learningcited by 938
  11. 2015Empirical Evaluation of Rectified Activations in Convolutional Networkcited by 515
  12. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
17 more
  1. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  2. 2016On the Expressive Power of Deep Neural Networkscited by 302
  3. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  4. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  5. 2022Black-Box Tuning for Language-Model-as-a-Servicecited by 254
  6. 2016Benefits of depth in neural networkscited by 248
  7. 2021SPoT: Better Frozen Model Adaptation through Soft Prompt Transfercited by 235
  8. 2022Relative representations enable zero-shot latent space communicationcited by 175
  9. 2018A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddingscited by 145
  10. 2023Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learningcited by 115
  11. 2022ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Promptscited by 85
  12. 2021On Transferability of Prompt Tuning for Natural Language Processingcited by 74
  13. 2021Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Promptscited by 64
  14. 2022ASIF: Coupled Data Turns Unimodal Models to Multimodal Without Trainingcited by 45
  15. 2021A Closer Look at How Fine-tuning Changes BERTcited by 42
  16. 2023Can discrete information extraction prompts generalize across language models?cited by 8
  17. 2023The Emergence of Chunking Structures with Hierarchical RNNcited by 2

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 5.

7 papers.

  1. 2025Adapt Once, Thrive with Updates: Transferable Parameter-Efficient Fine-Tuning on Evolving Base Modelscited by 5
  2. 2025TokMem: One-Token Procedural Memory for Large Language Modelscited by 5
  3. 2025LoRASuite: Efficient LoRA Adaptation Across Large Language Model Upgradescited by 4
  4. 2024Anonymization Prompt Learning for Facial Privacy-Preserving Text-to-Image Generationcited by 1
  5. 2025Efficient and Privacy-Preserving Soft Prompt Transfer for LLMscited by 1
  6. 2025Order-Level Attention Similarity Across Language Models: A Latent Commonalitycited by 1
  7. 2024Understanding Visual Concepts Across Modelscited by 0