mlsanityData to 7 Oct 2026

Zero-shot Cross-lingual Transfer of Prompt-based Tuning with a Unified Multilingual Prompt

arXiv 2202.11451 ↗First posted 23 Feb 2022cs.CLcited by 25

Not on the map. Why?

Abstract (arXiv)

Prompt-based tuning has been proven effective for pretrained language models (PLMs). While most of the existing work focuses on the monolingual prompts, we study the multilingual prompts for multilingual PLMs, especially in the zero-shot cross-lingual setting. To alleviate the effort of designing different prompts for multiple languages, we propose a novel model that uses a unified prompt for all languages, called UniPrompt. Different from the discrete prompts and soft prompts, the unified prompt is model-based and language-agnostic. Specifically, the unified prompt is initialized by a multilingual PLM to produce language-independent representation, after which is fused with the text input. During inference, the prompts can be pre-computed so that no extra computation cost is needed. To collocate with the unified prompt, we propose a new initialization method for the target label word to further improve the model's transferability across languages. Extensive experiments show that our proposed methods can significantly outperform the strong baselines across different languages. We release data and code to facilitate future research.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 18 of its 21 references that are in the map. First posted on arXiv 23 Feb 2022.

Builds on

18 of 21 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  8. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  9. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  10. 2021WARP: Word-level Adversarial ReProgrammingcited by 220
  11. 2021How Many Data Points is a Prompt Worth?cited by 197
  12. 2020SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddingscited by 155
6 more
  1. 2020The Multilingual Amazon Reviews Corpuscited by 142
  2. 2020Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?cited by 127
  3. 2021Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learnerscited by 115
  4. 2021Language Models are Few-shot Multilingual Learnerscited by 92
  5. 2021Discrete and Soft Prompting for Multilingual Modelscited by 49
  6. 2020A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matterscited by 45

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 10; 2024: 9; 2025: 2.

25 papers.

Sort:

By area:

  1. 2024Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languagescited by 36
  2. 2022Zero-Shot Rumor Detection with Propagation Structure via Prompt Learningcited by 32
  3. 2024Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Modelscited by 31
  4. 2023InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understandingcited by 29
  5. 2022Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languagescited by 17
  6. 2023From Classification to Generation: Insights into Crosslingual Retrieval Augmented ICLcited by 17
  7. 2022Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Modelscited by 16
  8. 2023RAMP: Retrieval and Attribute-Marking Enhanced Prompting for Attribute-Controlled Translationcited by 9
  9. 2023Bridging the Language Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMscited by 8
  10. 2022Multilingual Relation Classification via Efficient and Effective Promptingcited by 7
  11. 2023Is Prompt-Based Finetuning Always Better than Vanilla Finetuning? Insights from Cross-Lingual Language Understandingcited by 7
  12. 2023Crosslingual Retrieval Augmented In-context Learning for Banglacited by 7
13 more
  1. 2024ToPro: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Taskscited by 7
  2. 2024English Prompts are Better for NLI-based Zero-Shot Emotion Classification than Target-Language Promptscited by 7
  3. 2024Soft Prompt Tuning for Cross-Lingual Transfer: When Less is Morecited by 7
  4. 2024Learning from Contrastive Prompts: Automated Optimization and Adaptationcited by 7
  5. 2024The Roles of English in Evaluating Multilingual Language Modelscited by 7
  6. 2024Soft Language Prompts for Language Transfercited by 5
  7. 2025Adapting Whisper for Parameter-efficient Code-Switching Speech Recognition via Soft Prompt Tuningcited by 5
  8. 2023Soft Language Clustering for Multilingual Model Pre-trainingcited by 4
  9. 2023Unleashing the Multilingual Encoder Potential: Boosting Zero-Shot Performance via Probability Calibrationcited by 2
  10. 2023On the Analysis of Cross-Lingual Prompt Tuning for Decoder-based Multilingual Modelcited by 2
  11. 2023Multilingual Few-Shot Learning via Language Model Retrievalcited by 1
  12. 2024Bridging the Language Gap: Enhancing Multilingual Prompt-Based Code Generation in LLMs via Zero-Shot Cross-Lingual Transfercited by 1
  13. 2025Cross-Prompt Encoder for Low-Performing Languagescited by 0