Enhancing Cross-lingual Prompting with Dual Prompt Augmentation
Not on the map. Why?
Abstract (arXiv)
Prompting shows promising results in few-shot scenarios. However, its strength for multilingual/cross-lingual problems has not been fully exploited. Zhao and Sch"utze (2021) made initial explorations in this direction by presenting that cross-lingual prompting outperforms cross-lingual finetuning. In this paper, we conduct an empirical exploration on the effect of each component in cross-lingual prompting and derive language-agnostic Universal Prompting, which helps alleviate the discrepancies between source-language training and target-language inference. Based on this, we propose DPA, a dual prompt augmentation framework, aiming at relieving the data scarcity issue in few-shot cross-lingual prompting. Notably, for XNLI, our method achieves 46.54% with only 16 English training examples per class, significantly better than 34.99% of finetuning. Our code is available at https://github.com/DAMO-NLP-SG/DPA.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 28 of its 45 references that are in the map. First posted on arXiv 15 Feb 2022.
Builds on
28 of 45 references.
References · 28
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
- 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
- 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
- 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
- 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
- 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
- 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
16 more
- 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
- 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
- 2021True Few-Shot Learning with Language Modelscited by 343
- 2021Few-shot Learning with Multilingual Language Modelscited by 278
- 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 262
- 2020On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselinescited by 254
- 2020Revisiting Few-sample BERT Fine-tuningcited by 237
- 2021How Many Data Points is a Prompt Worth?cited by 197
- 2020MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classificationcited by 186
- 2021Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Modelscited by 139
- 2019Augmenting Data with Mixup for Sentence Classification: An Empirical Studycited by 125
- 2021Discrete and Soft Prompting for Multilingual Modelscited by 49
- 2020A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matterscited by 45
- 2021FlipDA: Effective and Robust Data Augmentation for Few-Shot Learningcited by 32
- 2020Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Modelcited by 24
- 2021MixUp Training Leads to Reduced Overfitting and Improved Calibration for the Transformer Architecturecited by 11
Built on by
Citing papers per year: 2023: 1; 2024: 2.
3 papers.