mlsanityData to 14 Oct 2026

Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Models

arXiv 2210.12360 ↗First posted 22 Oct 2022cs.CL cs.AIcited by 16

Not on the map. Why?

Abstract (arXiv)

Pre-trained multilingual language models show significant performance gains for zero-shot cross-lingual model transfer on a wide range of natural language understanding (NLU) tasks. Previously, for zero-shot cross-lingual evaluation, pre-trained models are only fine-tuned on English data and tested on a variety of target languages. In this paper, we do cross-lingual evaluation on various NLU tasks (sentence classification, sequence labeling, question answering) using prompt-tuning and compare it with fine-tuning. The results show that prompt tuning achieves much better cross-lingual transfer than fine-tuning across datasets, with only 0.1% to 0.3% tuned parameters. Additionally, we demonstrate through the analysis that prompt tuning can have better cross-lingual transferability of representations on downstream tasks with better aligned decision boundaries.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 23 of its 31 references that are in the map. First posted on arXiv 22 Oct 2022.

Builds on

23 of 31 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  7. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  8. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  9. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
  10. 2019How multilingual is Multilingual BERT?cited by 645
  11. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
  12. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 528
11 more
  1. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 488
  2. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  3. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  4. 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 264
  5. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 222
  6. 2021WARP: Word-level Adversarial ReProgrammingcited by 221
  7. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 168
  8. 2021Language Models are Few-shot Multilingual Learnerscited by 92
  9. 2021Discrete and Soft Prompting for Multilingual Modelscited by 49
  10. 2020VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generationcited by 34
  11. 2022Zero-shot Cross-lingual Transfer of Prompt-based Tuning with a Unified Multilingual Promptcited by 25

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 5; 2024: 8; 2025: 2.

16 papers.

Sort:

By area:

  1. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
  2. 2022BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Promptingcited by 90
  3. 2024Facing the Elephant in the Room: Visual Prompt Tuning or Full Finetuning?cited by 31
  4. 2024Enhancing Domain Adaptation through Prompt Gradient Alignmentcited by 8
  5. 2023Is Prompt-Based Finetuning Always Better than Vanilla Finetuning? Insights from Cross-Lingual Language Understandingcited by 7
  6. 2024ToPro: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Taskscited by 7
  7. 2024English Prompts are Better for NLI-based Zero-Shot Emotion Classification than Target-Language Promptscited by 7
  8. 2023Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuningcited by 5
  9. 2024Analysis of Multi-Source Language Training in Cross-Lingual Transfercited by 5
  10. 2023Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-Attentioncited by 4
  11. 2024UniBridge: A Unified Approach to Cross-Lingual Transfer Learning for Low-Resource Languagescited by 4
  12. 2023On the Analysis of Cross-Lingual Prompt Tuning for Decoder-based Multilingual Modelcited by 2
4 more
  1. 2023Prompt Tuned Embedding Classification for Multi-Label Industry Sector Allocationcited by 1
  2. 2025Statement-Tuning Enables Efficient Cross-lingual Generalization in Encoder-only Modelscited by 1
  3. 2024Prompt-based Personality Profiling: Reinforcement Learning for Relevance Filteringcited by 0
  4. 2025Zero-Shot Cross-Lingual Transfer using Prefix-Based Adaptationcited by 0