mlsanityData to 7 Oct 2026

Prototypical Fine-tuning: Towards Robust Performance Under Varying Data Sizes

arXiv 2211.13638 ↗First posted 24 Nov 2022cs.CL cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

In this paper, we move towards combining large parametric models with non-parametric prototypical networks. We propose prototypical fine-tuning, a novel prototypical framework for fine-tuning pretrained language models (LM), which automatically learns a bias to improve predictive performance for varying data sizes, especially low-resource settings. Our prototypical fine-tuning approach can automatically adjust the model capacity according to the number of data points and the model's inherent attributes. Moreover, we propose four principles for effective prototype fine-tuning towards the optimal solution. Experimental results across various datasets show that our work achieves significant performance improvements under various low-resource settings, as well as comparable and usually better performances in high-resource scenarios.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 40 of its 57 references that are in the map. First posted on arXiv 24 Nov 2022.

Builds on

40 of 57 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  7. 2016Matching Networks for One Shot Learningcited by 2,246
  8. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  9. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  10. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  11. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  12. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
28 more
  1. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  2. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  3. 2015Deep Metric Learning via Lifted Structured Feature Embeddingcited by 449
  4. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  5. 2021True Few-Shot Learning with Language Modelscited by 343
  6. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 275
  7. 2017Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictionscited by 234
  8. 2017Unsupervised Neural Machine Translationcited by 148
  9. 2017Data Augmentation for Low-Resource Neural Machine Translationcited by 139
  10. 2018Robust Classification with Convolutional Prototype Learningcited by 122
  11. 2019Infinite Mixture Prototypes for Few-Shot Learningcited by 99
  12. 2016Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classificationcited by 84
  13. 2021Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuningcited by 82
  14. 2015The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classificationcited by 76
  15. 2019Interpretable and Steerable Sequence Learning via Prototypescited by 72
  16. 2022GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspectivecited by 72
  17. 2021Variational Information Bottleneck for Effective Low-Resource Fine-Tuningcited by 51
  18. 2022NoisyTune: A Little Noise Can Help You Finetune Pretrained Language Models Bettercited by 43
  19. 2018Low-shot Learning via Covariance-Preserving Adversarial Augmentation Networkscited by 35
  20. 2021Towards Fine-Grained Reasoning for Fake News Detectioncited by 32
  21. 2020Analysis of Q-learning with Adaptation and Momentum Restart for Gradient Descentcited by 26
  22. 2022Self-explaining deep models with logic rule reasoningcited by 13
  23. 2022Code Recommendation for Open Source Software Developerscited by 13
  24. 2020Discrete Latent Variable Representations for Low-Resource Text Classificationcited by 10
  25. 2022Grad2Task: Improved Few-shot Text Classification Using Gradients for Task Representationcited by 8
  26. 2022Adversarial Contrastive Learning for Evidence-aware Fake News Detection with Graph Neural Networkscited by 7
  27. 2014Incremental Clustering: The Case for Extra Clusterscited by 5
  28. 2022A Mutually Reinforced Framework for Pretrained Sentence Embeddingscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 3; 2025: 3; 2026: 1.

10 papers.

  1. 2023Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queriescited by 82
  2. 2023Continual Learning on Dynamic Graphs via Parameter Isolationcited by 35
  3. 2024Prototypical Reward Network for Data-Efficient RLHFcited by 22
  4. 2023TMac: Temporal Multi-Modal Graph Learning for Acoustic Event Classificationcited by 9
  5. 2025Examining and Adapting Time for Multilingual Classification via Mixture of Temporal Expertscited by 9
  6. 2024Borrowing Treasures from Neighbors: In-Context Learning for Multimodal Learning with Missing Modalities and Data Scarcitycited by 5
  7. 2025ProtoTS: Learning Hierarchical Prototypes for Explainable Time Series Forecastingcited by 2
  8. 2024Generative Pre-trained Ranking Model with Over-parameterization at Web-Scale (Extended Abstract)cited by 0
  9. 2025ProtoBERT-LoRA: Parameter-Efficient Prototypical Finetuning for Immunotherapy Study Identificationcited by 0
  10. 2026Causal Connections: Leveraging Multilingual Fine-Tuning for Financial QA@FinCausal 2026cited by 0