mlsanityData to 13 Oct 2026

A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters

arXiv 2012.15682 ↗First posted 31 Dec 2020cs.CLcited by 45

Not on the map. Why?

Abstract (arXiv)

Few-shot crosslingual transfer has been shown to outperform its zero-shot counterpart with pretrained encoders like multilingual BERT. Despite its growing popularity, little to no attention has been paid to standardizing and analyzing the design of few-shot experiments. In this work, we highlight a fundamental risk posed by this shortcoming, illustrating that the model exhibits a high degree of sensitivity to the selection of few shots. We conduct a large-scale experimental study on 40 sets of sampled few shots for six diverse NLP tasks across up to 40 languages. We provide an analysis of success and failure cases of few-shot transfer, which highlights the role of lexical features. Additionally, we show that a straightforward full model finetuning approach is quite effective for few-shot transfer, outperforming several state-of-the-art few-shot approaches. As a step towards standardizing few-shot crosslingual experimental designs, we make our sampled few shots publicly available.

Key facts. Cited by 45 papers in the map (data through 13 Oct 2026). First posted on arXiv 31 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 6; 2022: 15; 2023: 16; 2024: 4; 2025: 2; 2026: 2.

45 papers.

Sort:

  1. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 687
  2. 2023BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfercited by 68
  3. 2024IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languagescited by 55
  4. 2022IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languagescited by 54
  5. 2021Discrete and Soft Prompting for Multilingual Modelscited by 49
  6. 2023XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languagescited by 34
  7. 2022Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifoldcited by 29
  8. 2021ConvFiT: Conversational Fine-Tuning of Pretrained Language Modelscited by 25
  9. 2022Zero-shot Cross-lingual Transfer of Prompt-based Tuning with a Unified Multilingual Promptcited by 25
  10. 2022QAmeleon: Multilingual QA with Only 5 Examplescited by 25
  11. 2021On Cross-Lingual Retrieval with Multilingual Text Encoderscited by 22
  12. 2023Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solutioncited by 22
33 more
  1. 2022MEAL: Stable and Active Learning for Few-Shot Promptingcited by 20
  2. 2023Transfer to a Low-Resource Language via Close Relatives: The Case Study on Faroesecited by 15
  3. 2023A Survey on Stability of Learning with Limited Labelled Data and its Sensitivity to the Effects of Randomnesscited by 14
  4. 2023In-Context Learning Dynamics with Random Binary Sequencescited by 13
  5. 2022Meta-Learning a Cross-lingual Manifold for Semantic Parsingcited by 11
  6. 2024Key ingredients for effective zero-shot cross-lingual knowledge transfer in generative taskscited by 11
  7. 2021LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Frameworkcited by 9
  8. 2022Delving Deeper into Cross-lingual Visual Question Answeringcited by 8
  9. 2023Free Lunch: Robust Cross-Lingual Transfer via Model Checkpoint Averagingcited by 8
  10. 2022Data-Efficient Strategies for Expanding Hate Speech Detection into Under-Resourced Languagescited by 7
  11. 2023To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languagescited by 7
  12. 2022Por Qu'e N~ao Utiliser Alla Spraak? Mixed Training with Gradient Optimization in Few-Shot Cross-Lingual Transfercited by 6
  13. 2022How to Fine-tune Models with Few Samples: Update, Data Augmentation, and Test-time Augmentationcited by 6
  14. 2024Adaptive Cross-lingual Text Classification through In-Context One-Shot Demonstrationscited by 6
  15. 2021TraVLR: Now You See It, Now You Don't! A Bimodal Dataset for Evaluating Visio-Linguistic Reasoningcited by 4
  16. 2022Zero-shot Cross-lingual Transfer is Under-specified Optimizationcited by 4
  17. 2023Prompt Learning to Mitigate Catastrophic Forgetting in Cross-lingual Transfer for Open-domain Dialogue Generationcited by 4
  18. 2022Enhancing Cross-lingual Prompting with Dual Prompt Augmentationcited by 3
  19. 2025Language Fusion for Parameter-Efficient Cross-lingual Transfercited by 3
  20. 2022Cross-lingual Transfer Learning for Check-worthy Claim Identification over Twittercited by 2
  21. 2023Optimal Transport Posterior Alignment for Cross-lingual Semantic Parsingcited by 2
  22. 2022Searching for Effective Multilingual Fine-Tuning Methods: A Case Study in Summarizationcited by 1
  23. 2023Measuring Catastrophic Forgetting in Cross-Lingual Transfer Paradigms: Exploring Tuning Strategiescited by 1
  24. 2023Vicinal Risk Minimization for Few-Shot Cross-lingual Transfer in Abusive Language Detectioncited by 1
  25. 2024On Sensitivity of Learning with Limited Labelled Data to the Effects of Randomness: Impact of Interactions and Systematic Choicescited by 1
  26. 2022Massively Multilingual Lexical Specialization of Multilingual Transformerscited by 0
  27. 2023Cross-lingual German Biomedical Information Extraction: from Zero-shot to Human-in-the-Loopcited by 0
  28. 2023A General-Purpose Multilingual Document Encodercited by 0
  29. 2023How to Solve Few-Shot Abusive Content Detection Using the Data We Actually Havecited by 0
  30. 2023Empirical study of pretrained multilingual language models for zero-shot cross-lingual knowledge transfer in generationcited by 0
  31. 2025Data Augmentation for Maltese NLP using Transliterated and Machine Translated Arabic Datacited by 0
  32. 2026How Much Noise Can BERT Handle? Insights from Multilingual Sentence Difficulty Detectioncited by 0
  33. 2026YoNER: A New Yor`ub'a Multi-domain Named Entity Recognition Datasetcited by 0